Schedule Builtins
All schedule builtins operate on Polars DataFrames in ctx.datasets["filename.txt"].
MatchCondition
Used by RemoveRows, UpdateFields, and ClearField to select which rows to operate on.
from continuous_gtfs.builtins.schedule import MatchCondition
# Exact match
MatchCondition("route_id", value="100479")
# Regex match
MatchCondition("service_id", regex=r"^LLR.*")
Combining conditions
Conditions combine as groups:
- A flat list is one group — every condition must match (all-of).
- A list of lists is several groups — every condition within a group must match, and a row is selected if it matches any group (any-of).
# all-of: calendars with no service on any weekday
[
MatchCondition("monday", value="0"),
MatchCondition("tuesday", value="0"),
]
# any-of: three stops in one step instead of three steps
[
[MatchCondition("stop_id", value="N13")],
[MatchCondition("stop_id", value="N13-T1")],
[MatchCondition("stop_id", value="N13-T2")],
]
# (A and B) or C
[
[MatchCondition("route_id", value="2LINE"), MatchCondition("shape_id", value="E27:E09")],
[MatchCondition("route_id", value="TLINE")],
]
A single condition means the same thing flat or as a one-element group. Don't mix the two shapes in one list, and don't pass an empty group — both raise ValueError when the step is constructed.
Missing columns
If any condition names a column the file doesn't have, the step leaves that file untouched and emits a column_missing warning finding (one per missing column, with the file and column in its context) so the misconfiguration surfaces in the run's issues instead of passing silently. An empty conditions list is a plain no-op with no finding.
RemoveRows
Delete rows matching conditions from a GTFS file.
from continuous_gtfs.builtins.schedule import RemoveRows, MatchCondition
remove_route = RemoveRows(
"routes.txt", # which file
[MatchCondition("route_id", value="OLD_ROUTE")], # match conditions (flat = all-of)
description="Remove deprecated route", # optional
)
# Any-of: collapse per-stop steps into one
remove_n13_family = RemoveRows(
"stops.txt",
[
[MatchCondition("stop_id", value="N13")],
[MatchCondition("stop_id", value="N13-T1")],
[MatchCondition("stop_id", value="N13-T2")],
],
description="Remove non-revenue stop N13 and its platforms",
)
With exclude
Protect specific rows from removal:
# Remove all LLR services, EXCEPT the Spring 2026 one
remove_llr = RemoveRows(
"calendar.txt",
[MatchCondition("service_id", regex=r"^LLR.*")],
exclude=[
[MatchCondition("service_id", value="LLR_SP26")],
],
)
exclude takes the same group shape as conditions (all-of within a group, any-of across groups); a row matching any exclude group is protected. Include and exclude are symmetric.
Edge cases
- If the file doesn't exist in
ctx.datasets, the step is a no-op. - If a condition — in
conditionsorexclude— references a column that doesn't exist, the step leaves the file untouched and emits acolumn_missingwarning (see Missing columns).
UpdateFields
Modify field values on rows matching conditions.
from continuous_gtfs.builtins.schedule import UpdateFields, MatchCondition
update_2line = UpdateFields(
"routes.txt",
[MatchCondition("route_id", value="2LINE")],
{
"route_long_name": "South Bellevue - Downtown Redmond",
"route_color": "007CAD",
"route_text_color": "FFFFFF",
},
description="Update 2 Line route metadata",
)
Only columns that already exist in the DataFrame are updated. Unknown column names in the updates dict are silently ignored.
ClearField
Set a field to empty string on matching rows. Convenience wrapper around UpdateFields for the common case.
from continuous_gtfs.builtins.schedule import ClearField, MatchCondition
clear_blocks = ClearField(
"trips.txt",
"block_id",
[MatchCondition("route_id", value="SNDR_TL")],
description="Clear block_id for Sounder Tacoma-Lakewood",
)
UpdateFeedInfo
Update feed_info.txt metadata fields. Applies to all rows (no conditions needed).
from continuous_gtfs.builtins.schedule import UpdateFeedInfo
update_feed = UpdateFeedInfo(
publisher_name="Sound Transit",
publisher_url="https://soundtransit.org",
feed_lang="en",
)
All parameters are optional — only specified fields are updated.
SortRows
Reorder a file's rows by one or more fields. Use it to pin the row order of any file whose upstream steps (a supplemental merge, a dedup) leave rows in an order that varies between runs — a sorted file's bytes, and so its content-hash version, depend only on its content.
from continuous_gtfs.builtins.schedule import SortKey, SortRows
sort_stop_times = SortRows(
"stop_times.txt",
["trip_id", SortKey("stop_sequence", numeric=True)],
description="Order stop_times by trip, then stop sequence",
)
Each entry in by is a SortKey(field, descending=False, numeric=False); a bare string is shorthand for an ascending text key.
- GTFS columns are text, so
10sorts before9unless the key isnumeric.numeric=Truecompares as numbers for ordering only — the written values are unchanged. - Empty string and null are the same value and sort last in either direction, as do values that fail a
numericcast. - After your keys, every remaining tie is broken by all columns of the file in header order (ascending, as text), so the output order never depends on the order rows arrived in.
- The step defaults to
after="*"so it runs after every other step that touches the file. Pass an explicitafter=[...]to override.
Edge cases
- If the file doesn't exist in
ctx.datasets, the step is a no-op. - If a sort field is missing from the file, the file is left unsorted and the step emits a
warningfinding (column_missing) for each missing field. - Values that aren't numeric under a
numerickey emit onesort_key_not_numericfinding whose occurrence count is the number of such values.
Semantic removals
RemoveRows edits one file and knows nothing about GTFS: remove a station's row from stops.txt and its platforms still say parent_station=N15, stop_times.txt still calls at them. The four semantic removal steps remove a record with everything the GTFS model says belongs to it, then tidy what the removal left empty — and report every file they touched.
| Step | Removes | With it |
|---|---|---|
RemoveRoutes |
rows of routes.txt |
their trips (and everything a trip takes), fare_rules, route_networks, attributions, transfers |
RemoveTrips |
rows of trips.txt |
stop_times, frequencies, attributions, transfers |
RemoveStops |
rows of stops.txt |
stop_times, transfers, pathways, stop_areas, and the stop's parent_station children |
RemoveServices |
a service in both calendar.txt and calendar_dates.txt |
its trips (and everything a trip takes) |
Selecting records
RemoveRoutes, RemoveTrips, and RemoveStops take the same conditions / exclude groups as RemoveRows, evaluated on the record's home file, plus an ids=[...] shortcut — a list of key values, the same as one any-of group per id. A step with neither raises at construction.
from continuous_gtfs.builtins.schedule import MatchCondition, RemoveRoutes, RemoveStops
remove_sounder = RemoveRoutes(ids=["SNDR_TL", "SNDR_EV"])
remove_shuttles = RemoveRoutes(
[MatchCondition("route_short_name", regex=r"Shuttle$")],
exclude=[[MatchCondition("route_id", value="ALS-FWD")]],
)
RemoveStops and parent_station
Removing a station means the station. By default (children="remove") the rows whose parent_station names a removed stop go too, recursively — platforms, entrances, generic nodes, boarding areas — and their stop_times with them. children="detach" keeps platforms as standalone stops (their parent_station is cleared) and still removes entrances and nodes, which cannot exist without a parent.
# sound-transit#251: the whole Shoreline South/148th station
remove_n15 = RemoveStops(ids=["N15"])
# Demote the station instead: platforms stay, entrances go
detach_n15 = RemoveStops(ids=["N15"], children="detach")
A trip whose every stop_times row was at a removed stop is removed as well, and the step emits a trips_removed_with_stops warning with the count, so a platform removal that took trips with it is visible in the run's issues. Set empty_trips="keep" (or "warn") to leave such trips in place.
What the removal leaves empty
A removal can leave a route, a service, or a shape with no trips. Each step decides per record type with a tri-state parameter — "remove", "warn" (keep the record; emit a warning naming it), or "keep" — and only for records this step emptied; a route that already had no trips is not its concern.
| Step | empty_trips |
empty_services |
empty_routes |
empty_shapes |
|---|---|---|---|---|
RemoveRoutes |
— | remove |
— | warn |
RemoveTrips |
— | remove |
warn |
warn |
RemoveStops |
remove |
remove |
warn |
warn |
RemoveServices |
— | — | warn |
warn |
A service with no trips is dead data and goes. A route is a public identity agencies keep on purpose (a bus bridge), so it stays with a route_without_trips warning. A shape is display geometry another trip may be re-pointed at, so it is never removed by default — shape_without_trips says which ones are now unused; pass empty_shapes="remove" to drop them.
RemoveServices
Selection is by service_id only — ids=[...] and/or condition groups whose every condition is on service_id, evaluated over the union of both calendar files so a regex reaches a service defined only in calendar_dates.txt. A condition on any other column raises at construction. Three selectors add to the selection; they union, and exclude protects a service from all of them:
never_active=Trueresolves each service's actual dates (weekday flags over the date range, thencalendar_dates.txtadds and removals) and removes every service with no active date at all — an all-zero calendar row with no added dates, or a calendar row whose every service day is cancelled by exceptions. This is what "remove inactive calendars" means once exceptions are taken into account.simplify_calendar(defaultTrue) goes with it: a service that has an all-zerocalendar.txtrow but runs oncalendar_dates.txtadds is kept, and its dead calendar row is dropped. Passsimplify_calendar=Falseto leave that row alone.expired_beforeremoves every service whose last active date is strictly before a cutoff — the expired service periods, with their trips. PassTrueto use the feed's ownfeed_info.txtfeed_start_date, or an explicit"YYYYMMDD"string (ordatetime.date) to override it. A pastend_datealone does not expire a service that acalendar_dates.txtadd keeps alive on or after the cutoff. Exception rows dated before the cutoff on services that survive are removed as well — a cancellation for a day already served is dead data. A service with no active date at all isnever_active's; combine the two when you want both.
from continuous_gtfs.builtins.schedule import MatchCondition, RemoveServices
drop_dead_services = RemoveServices(never_active=True)
drop_llr = RemoveServices(
[MatchCondition("service_id", regex=r"^LLR")],
exclude=[[MatchCondition("service_id", value="LLR_SP26")]],
)
# Everything that stopped running before this feed's feed_start_date
drop_expired = RemoveServices(expired_before=True)
# Or pin the cutoff — and take the never-active calendars in the same step
drop_pre_fall = RemoveServices(expired_before="20260901", never_active=True)
expired_before=True fails the step when the feed has no feed_info.txt or no feed_start_date — a feed without one and no configured cutoff is a data or configuration error, and the step will not guess. It never uses the current date: the same feed and configuration must produce the same output whenever the pipeline runs.
Findings
Every semantic removal reports what it did:
rows_removed(info) — one per touched file, the root file included, withoccurrence_countthe rows removed and contextfile,record_type, androle(root,cascade, orprune).route_without_trips/shape_without_trips(warning) — one per record left empty under"warn".trips_removed_with_stops(warning) — fromRemoveStops, when emptied trips were removed.column_missing(warning) — a condition named a column the home file lacks; the step did nothing.
Every removed route, trip, stop, service, or shape is also recorded in ctx.id_mappings with a None value (services under calendar.txt), so a custom step downstream can read what went away — see ID Mappings.
Edge cases
- A missing home file, or a selection that matches nothing, is a silent no-op.
- A referencing file that is absent, or lacks the referencing column, is skipped.
- References not in the tables above — fare v2,
zone_id,translations.txt,level_id— are left alone.
Common parameters
All schedule builtins accept these keyword arguments:
| Parameter | Type | Default | Description |
|---|---|---|---|
after |
list[Step] |
[] |
Steps that must run before this one |
before |
list[Step] |
[] |
Steps that must run after this one |
priority |
int |
100 |
Tiebreak for steps at the same DAG level (lower = earlier) |
enabled |
bool |
True |
Set to False to skip this step |
tags |
list[str] |
[] |
Arbitrary labels for categorization |
data_owner |
str |
None |
Email of responsible staff member |
description |
str |
auto-generated | Human-readable label for DAG display |