Skip to content

Schedule Builtins

All schedule builtins operate on Polars DataFrames in ctx.datasets["filename.txt"].

MatchCondition

Used by RemoveRows, UpdateFields, and ClearField to select which rows to operate on.

from continuous_gtfs.builtins.schedule import MatchCondition

# Exact match
MatchCondition("route_id", value="100479")

# Regex match
MatchCondition("service_id", regex=r"^LLR.*")

Combining conditions

Conditions combine as groups:

  • A flat list is one group — every condition must match (all-of).
  • A list of lists is several groups — every condition within a group must match, and a row is selected if it matches any group (any-of).
# all-of: calendars with no service on any weekday
[
    MatchCondition("monday", value="0"),
    MatchCondition("tuesday", value="0"),
]

# any-of: three stops in one step instead of three steps
[
    [MatchCondition("stop_id", value="N13")],
    [MatchCondition("stop_id", value="N13-T1")],
    [MatchCondition("stop_id", value="N13-T2")],
]

# (A and B) or C
[
    [MatchCondition("route_id", value="2LINE"), MatchCondition("shape_id", value="E27:E09")],
    [MatchCondition("route_id", value="TLINE")],
]

A single condition means the same thing flat or as a one-element group. Don't mix the two shapes in one list, and don't pass an empty group — both raise ValueError when the step is constructed.

Missing columns

If any condition names a column the file doesn't have, the step leaves that file untouched and emits a column_missing warning finding (one per missing column, with the file and column in its context) so the misconfiguration surfaces in the run's issues instead of passing silently. An empty conditions list is a plain no-op with no finding.

RemoveRows

Delete rows matching conditions from a GTFS file.

from continuous_gtfs.builtins.schedule import RemoveRows, MatchCondition

remove_route = RemoveRows(
    "routes.txt",                                    # which file
    [MatchCondition("route_id", value="OLD_ROUTE")], # match conditions (flat = all-of)
    description="Remove deprecated route",           # optional
)

# Any-of: collapse per-stop steps into one
remove_n13_family = RemoveRows(
    "stops.txt",
    [
        [MatchCondition("stop_id", value="N13")],
        [MatchCondition("stop_id", value="N13-T1")],
        [MatchCondition("stop_id", value="N13-T2")],
    ],
    description="Remove non-revenue stop N13 and its platforms",
)

With exclude

Protect specific rows from removal:

# Remove all LLR services, EXCEPT the Spring 2026 one
remove_llr = RemoveRows(
    "calendar.txt",
    [MatchCondition("service_id", regex=r"^LLR.*")],
    exclude=[
        [MatchCondition("service_id", value="LLR_SP26")],
    ],
)

exclude takes the same group shape as conditions (all-of within a group, any-of across groups); a row matching any exclude group is protected. Include and exclude are symmetric.

Edge cases

  • If the file doesn't exist in ctx.datasets, the step is a no-op.
  • If a condition — in conditions or exclude — references a column that doesn't exist, the step leaves the file untouched and emits a column_missing warning (see Missing columns).

UpdateFields

Modify field values on rows matching conditions.

from continuous_gtfs.builtins.schedule import UpdateFields, MatchCondition

update_2line = UpdateFields(
    "routes.txt",
    [MatchCondition("route_id", value="2LINE")],
    {
        "route_long_name": "South Bellevue - Downtown Redmond",
        "route_color": "007CAD",
        "route_text_color": "FFFFFF",
    },
    description="Update 2 Line route metadata",
)

Only columns that already exist in the DataFrame are updated. Unknown column names in the updates dict are silently ignored.

ClearField

Set a field to empty string on matching rows. Convenience wrapper around UpdateFields for the common case.

from continuous_gtfs.builtins.schedule import ClearField, MatchCondition

clear_blocks = ClearField(
    "trips.txt",
    "block_id",
    [MatchCondition("route_id", value="SNDR_TL")],
    description="Clear block_id for Sounder Tacoma-Lakewood",
)

UpdateFeedInfo

Update feed_info.txt metadata fields. Applies to all rows (no conditions needed).

from continuous_gtfs.builtins.schedule import UpdateFeedInfo

update_feed = UpdateFeedInfo(
    publisher_name="Sound Transit",
    publisher_url="https://soundtransit.org",
    feed_lang="en",
)

All parameters are optional — only specified fields are updated.

SortRows

Reorder a file's rows by one or more fields. Use it to pin the row order of any file whose upstream steps (a supplemental merge, a dedup) leave rows in an order that varies between runs — a sorted file's bytes, and so its content-hash version, depend only on its content.

from continuous_gtfs.builtins.schedule import SortKey, SortRows

sort_stop_times = SortRows(
    "stop_times.txt",
    ["trip_id", SortKey("stop_sequence", numeric=True)],
    description="Order stop_times by trip, then stop sequence",
)

Each entry in by is a SortKey(field, descending=False, numeric=False); a bare string is shorthand for an ascending text key.

  • GTFS columns are text, so 10 sorts before 9 unless the key is numeric. numeric=True compares as numbers for ordering only — the written values are unchanged.
  • Empty string and null are the same value and sort last in either direction, as do values that fail a numeric cast.
  • After your keys, every remaining tie is broken by all columns of the file in header order (ascending, as text), so the output order never depends on the order rows arrived in.
  • The step defaults to after="*" so it runs after every other step that touches the file. Pass an explicit after=[...] to override.

Edge cases

  • If the file doesn't exist in ctx.datasets, the step is a no-op.
  • If a sort field is missing from the file, the file is left unsorted and the step emits a warning finding (column_missing) for each missing field.
  • Values that aren't numeric under a numeric key emit one sort_key_not_numeric finding whose occurrence count is the number of such values.

Semantic removals

RemoveRows edits one file and knows nothing about GTFS: remove a station's row from stops.txt and its platforms still say parent_station=N15, stop_times.txt still calls at them. The four semantic removal steps remove a record with everything the GTFS model says belongs to it, then tidy what the removal left empty — and report every file they touched.

Step Removes With it
RemoveRoutes rows of routes.txt their trips (and everything a trip takes), fare_rules, route_networks, attributions, transfers
RemoveTrips rows of trips.txt stop_times, frequencies, attributions, transfers
RemoveStops rows of stops.txt stop_times, transfers, pathways, stop_areas, and the stop's parent_station children
RemoveServices a service in both calendar.txt and calendar_dates.txt its trips (and everything a trip takes)

Selecting records

RemoveRoutes, RemoveTrips, and RemoveStops take the same conditions / exclude groups as RemoveRows, evaluated on the record's home file, plus an ids=[...] shortcut — a list of key values, the same as one any-of group per id. A step with neither raises at construction.

from continuous_gtfs.builtins.schedule import MatchCondition, RemoveRoutes, RemoveStops

remove_sounder = RemoveRoutes(ids=["SNDR_TL", "SNDR_EV"])

remove_shuttles = RemoveRoutes(
    [MatchCondition("route_short_name", regex=r"Shuttle$")],
    exclude=[[MatchCondition("route_id", value="ALS-FWD")]],
)

RemoveStops and parent_station

Removing a station means the station. By default (children="remove") the rows whose parent_station names a removed stop go too, recursively — platforms, entrances, generic nodes, boarding areas — and their stop_times with them. children="detach" keeps platforms as standalone stops (their parent_station is cleared) and still removes entrances and nodes, which cannot exist without a parent.

# sound-transit#251: the whole Shoreline South/148th station
remove_n15 = RemoveStops(ids=["N15"])

# Demote the station instead: platforms stay, entrances go
detach_n15 = RemoveStops(ids=["N15"], children="detach")

A trip whose every stop_times row was at a removed stop is removed as well, and the step emits a trips_removed_with_stops warning with the count, so a platform removal that took trips with it is visible in the run's issues. Set empty_trips="keep" (or "warn") to leave such trips in place.

What the removal leaves empty

A removal can leave a route, a service, or a shape with no trips. Each step decides per record type with a tri-state parameter — "remove", "warn" (keep the record; emit a warning naming it), or "keep" — and only for records this step emptied; a route that already had no trips is not its concern.

Step empty_trips empty_services empty_routes empty_shapes
RemoveRoutes — remove — warn
RemoveTrips — remove warn warn
RemoveStops remove remove warn warn
RemoveServices — — warn warn

A service with no trips is dead data and goes. A route is a public identity agencies keep on purpose (a bus bridge), so it stays with a route_without_trips warning. A shape is display geometry another trip may be re-pointed at, so it is never removed by default — shape_without_trips says which ones are now unused; pass empty_shapes="remove" to drop them.

RemoveServices

Selection is by service_id only — ids=[...] and/or condition groups whose every condition is on service_id, evaluated over the union of both calendar files so a regex reaches a service defined only in calendar_dates.txt. A condition on any other column raises at construction. Three selectors add to the selection; they union, and exclude protects a service from all of them:

  • never_active=True resolves each service's actual dates (weekday flags over the date range, then calendar_dates.txt adds and removals) and removes every service with no active date at all — an all-zero calendar row with no added dates, or a calendar row whose every service day is cancelled by exceptions. This is what "remove inactive calendars" means once exceptions are taken into account.
  • simplify_calendar (default True) goes with it: a service that has an all-zero calendar.txt row but runs on calendar_dates.txt adds is kept, and its dead calendar row is dropped. Pass simplify_calendar=False to leave that row alone.
  • expired_before removes every service whose last active date is strictly before a cutoff — the expired service periods, with their trips. Pass True to use the feed's own feed_info.txt feed_start_date, or an explicit "YYYYMMDD" string (or datetime.date) to override it. A past end_date alone does not expire a service that a calendar_dates.txt add keeps alive on or after the cutoff. Exception rows dated before the cutoff on services that survive are removed as well — a cancellation for a day already served is dead data. A service with no active date at all is never_active's; combine the two when you want both.
from continuous_gtfs.builtins.schedule import MatchCondition, RemoveServices

drop_dead_services = RemoveServices(never_active=True)

drop_llr = RemoveServices(
    [MatchCondition("service_id", regex=r"^LLR")],
    exclude=[[MatchCondition("service_id", value="LLR_SP26")]],
)

# Everything that stopped running before this feed's feed_start_date
drop_expired = RemoveServices(expired_before=True)

# Or pin the cutoff — and take the never-active calendars in the same step
drop_pre_fall = RemoveServices(expired_before="20260901", never_active=True)

expired_before=True fails the step when the feed has no feed_info.txt or no feed_start_date — a feed without one and no configured cutoff is a data or configuration error, and the step will not guess. It never uses the current date: the same feed and configuration must produce the same output whenever the pipeline runs.

Findings

Every semantic removal reports what it did:

  • rows_removed (info) — one per touched file, the root file included, with occurrence_count the rows removed and context file, record_type, and role (root, cascade, or prune).
  • route_without_trips / shape_without_trips (warning) — one per record left empty under "warn".
  • trips_removed_with_stops (warning) — from RemoveStops, when emptied trips were removed.
  • column_missing (warning) — a condition named a column the home file lacks; the step did nothing.

Every removed route, trip, stop, service, or shape is also recorded in ctx.id_mappings with a None value (services under calendar.txt), so a custom step downstream can read what went away — see ID Mappings.

Edge cases

  • A missing home file, or a selection that matches nothing, is a silent no-op.
  • A referencing file that is absent, or lacks the referencing column, is skipped.
  • References not in the tables above — fare v2, zone_id, translations.txt, level_id — are left alone.

Common parameters

All schedule builtins accept these keyword arguments:

Parameter Type Default Description
after list[Step] [] Steps that must run before this one
before list[Step] [] Steps that must run after this one
priority int 100 Tiebreak for steps at the same DAG level (lower = earlier)
enabled bool True Set to False to skip this step
tags list[str] [] Arbitrary labels for categorization
data_owner str None Email of responsible staff member
description str auto-generated Human-readable label for DAG display