🙏 Question
How do versions for lines, routes, patterns and journeys work in practice?
Leonard Ehrenfried
I'm tasked with consuming a NeTEx feed that (claims to) adhere to EPIP. In it it contains several SERVICE JOURNEYs that are almost identical but the data producer says that they should be filtered out based on the version of their LINE.
Here are the service journeys:
Here are the service journeys:
<ServiceJourney responsibilitySetRef="it:apb:ResponsibilitySet:1_SAD_Bahn:" id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-2-25260:T0:" version="2">
<TransportMode>rail</TransportMode>
<DepartureTime>07:01:00</DepartureTime>
<dayTypes>
<DayTypeRef ref="it:apb:DayType:T0_6:" version="any"></DayTypeRef>
</dayTypes>
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a200505:" version="2"></ServiceJourneyPatternRef>
<VehicleTypeRef ref="it:apb:VehicleType:A1:" version="any"></VehicleTypeRef>
<trainNumbers>
<TrainNumberRef ref="it:apb:TrainNumber:7040:" version="any"></TrainNumberRef>
</trainNumbers>
<passingTimes>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-2-0050501:" version="2"></StopPointInJourneyPatternRef>
<DepartureTime>07:01:00</DepartureTime>
</TimetabledPassingTime>
...
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-2-0050518:" version="2"></StopPointInJourneyPatternRef>
<ArrivalTime>08:13:00</ArrivalTime>
</TimetabledPassingTime>
</passingTimes>
<facilities>
<ServiceFacilitySet id="it:apb:ServiceFacilitySet:032508S-SAD_Bahn-4-2-25260:T0:" version="any">
<AccommodationAccessList>freeSeating</AccommodationAccessList>
</ServiceFacilitySet>
</facilities>
</ServiceJourney>and the second one
<ServiceJourney responsibilitySetRef="it:apb:ResponsibilitySet:1_SAD_Bahn:" id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-3-25260:T0:" version="3">
<TransportMode>rail</TransportMode>
<DepartureTime>07:01:00</DepartureTime>
<dayTypes>
<DayTypeRef ref="it:apb:DayType:T0_6:" version="any"></DayTypeRef>
</dayTypes>
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a300505:" version="3"></ServiceJourneyPatternRef>
<VehicleTypeRef ref="it:apb:VehicleType:A1:" version="any"></VehicleTypeRef>
<trainNumbers>
<TrainNumberRef ref="it:apb:TrainNumber:7040:" version="any"></TrainNumberRef>
</trainNumbers>
<passingTimes>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050501:" version="3"></StopPointInJourneyPatternRef>
<DepartureTime>07:01:00</DepartureTime>
</TimetabledPassingTime>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050502:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>07:05:00</ArrivalTime>
<DepartureTime>07:05:00</DepartureTime>
</TimetabledPassingTime>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050503:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>07:10:00</ArrivalTime>
<DepartureTime>07:10:00</DepartureTime>
</TimetabledPassingTime>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050504:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>07:13:00</ArrivalTime>
<DepartureTime>07:13:00</DepartureTime>
</TimetabledPassingTime>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050505:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>07:17:00</ArrivalTime>
<DepartureTime>07:18:00</DepartureTime>
</TimetabledPassingTime>
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050506:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>07:25:00</ArrivalTime>
<DepartureTime>07:27:00</DepartureTime>
</TimetabledPassingTime>
...
<TimetabledPassingTime version="any">
<StopPointInJourneyPatternRef ref="it:apb:StopPointInJourneyPattern:03250S.25a-3-0050518:" version="3"></StopPointInJourneyPatternRef>
<ArrivalTime>08:13:00</ArrivalTime>
</TimetabledPassingTime>
</passingTimes>
<facilities>
<ServiceFacilitySet id="it:apb:ServiceFacilitySet:032508S-SAD_Bahn-4-3-25260:T0:" version="any">
<AccommodationAccessList>freeSeating</AccommodationAccessList>
</ServiceFacilitySet>
</facilities>
</ServiceJourney>You can see that they are very similar and they use the same DAY TYPE REF at the same version "any".
Now the data producer says that one should be excluded on certain dates based on its version: one is of version 2 and the other one of version 3.
They both refer to the same JOURNEY PATTERN at versions 2 and 3 respectively.
The SERVICE JOURNEY PATTERN exists only once in the file, at version 2.
<ServiceJourneyPattern id="it:apb:ServiceJourneyPattern:03250S.25a200505:" version="2">
<Name lang="de">5</Name>
<RouteRef ref="it:apb:Route:3-250-S-25a-2-5/H:" version="any"></RouteRef>
<pointsInSequence>
<StopPointInJourneyPattern id="it:apb:StopPointInJourneyPattern:03250S.25a-2-0050501:" version="2" order="1">
<ScheduledStopPointRef ref="it:apb:ScheduledStopPoint:it-22021-36-50-31002:" version="any"></ScheduledStopPointRef>
<ForAlighting>true</ForAlighting>
<ForBoarding>true</ForBoarding>
<DestinationDisplayRef ref="it:apb:DestinationDisplay:5007:" version="any"></DestinationDisplayRef>
<RequestStop>false</RequestStop>
<StopUse>access</StopUse>
</StopPointInJourneyPattern>
<StopPointInJourneyPattern id="it:apb:StopPointInJourneyPattern:03250S.25a-2-0050502:" version="2" order="2">
...It has a RouteRef of version "any".
The ROUTE looks like this - it doesn't have a version, just "any":
<Route id="it:apb:Route:3-250-S-25a-2-5/H:" version="any">
<LineRef ref="it:apb:Line:03250S.25a:" version="2"></LineRef>
<DirectionRef ref="it:apb:Direction:H:" version="any"></DirectionRef>
<pointsInSequence>
<PointOnRoute id="it:apb:PointOnRoute:3-250-S-25a-2-5/H_1:" version="any" order="1">
<RoutePointRef ref="it:apb:ScheduledStopPoint:it-22021-36-50-31002:" version="any"></RoutePointRef>
</PointOnRoute>
<PointOnRoute id="it:apb:PointOnRoute:3-250-S-25a-2-5/H_2:" version="any" order="2">
<RoutePointRef ref="it:apb:ScheduledStopPoint:it-22021-47-50-31021:" version="any"></RoutePointRef>
</PointOnRoute>
...So it in turn then refers to the LINE at version 2.
And indeed we can find several versions of that LINE with different validity periods:
<Line responsibilitySetRef="it:apb:ResponsibilitySet:1_SAD_Bahn:" id="it:apb:Line:03250S.25a:" version="2">
<ValidBetween>
<FromDate>2024-12-15T00:00:00</FromDate>
<ToDate>2025-01-07T23:59:59</ToDate>
</ValidBetween>
<Name lang="de">REG</Name>
<ShortName lang="de">REG</ShortName>
<TransportMode>rail</TransportMode>
<TransportSubmode>
<RailSubmode>regionalRail</RailSubmode>
</TransportSubmode>
<PublicCode>REG</PublicCode>
<PrivateCode>2508</PrivateCode>
<OperatorRef ref="it:apb:Operator:050:" version="any"></OperatorRef>
</Line>
<Line responsibilitySetRef="it:apb:ResponsibilitySet:1_SAD_Bahn:" id="it:apb:Line:03250S.25a:" version="3">
<ValidBetween>
<FromDate>2025-01-08T00:00:00</FromDate>
<ToDate>2025-02-15T23:59:59</ToDate>
</ValidBetween>
<Name lang="de">REG</Name>
<ShortName lang="de">REG</ShortName>
<TransportMode>rail</TransportMode>
<TransportSubmode>
<RailSubmode>regionalRail</RailSubmode>
</TransportSubmode>
<PublicCode>REG</PublicCode>
<PrivateCode>2508</PrivateCode>
<OperatorRef ref="it:apb:Operator:050:" version="any"></OperatorRef>
</Line>
<Line responsibilitySetRef="it:apb:ResponsibilitySet:1_SAD_Bahn:" id="it:apb:Line:03250S.25a:" version="5">
<ValidBetween>
<FromDate>2025-02-16T00:00:00</FromDate>
<ToDate>2025-10-25T23:59:59</ToDate>
</ValidBetween>
<Name lang="de">REG</Name>
<ShortName lang="de">REG</ShortName>
<TransportMode>rail</TransportMode>
<TransportSubmode>
<RailSubmode>regionalRail</RailSubmode>
</TransportSubmode>
<PublicCode>REG</PublicCode>
<PrivateCode>2508</PrivateCode>
<OperatorRef ref="it:apb:Operator:050:" version="any"></OperatorRef>
</Line>Now, the data producer says that the version of the SERVICE JOURNEY (2 and 3) should be used to select the validity period of the LINE a few layers up - basically the version becomes a sort of global selector.
I'm a bit skeptical about this statement but I cannot find a very good explanation in the EPIP or NeTEx standards either.
Can someone enlighten me if this is how you are supposed to use versions?
If you are trying to have some journeys run on a particular set of days, why don't you use a different operating period?
It seems strange that the LINE has several versions but the line itself doesn't change - it's the operating periods that change.
However, note that
the two mentioned ServiceJourney-element have different Ids and are defined for the same DayType, so they exist in parallel if they are part of the same delivery.
Generally the data-delivery should be rejected as a whole if any of the included ServiceJourney-versions are valid on any dates, during the validity-period of the containing VersionFrame, that the referenced Line is not valid. In our world it is a difference if you refer to a Line by Id or by Id + version.
In the first case it is ok as long as any version of the Line is valid on all the dates found by combining DayType with the Calender during the validity-period of the VersionFrame . In the later case the referred version of the Line must cover all the dates.
Regarding the spirit: none the EPIP-examples have been using the VDV-like ValidBetween. Hence Line-Version is invented by Mentz. A construct which is used in a Mentz system not 'simple for a generic data user' the EPIP-profile was targeted for. Yes, EPIP-materialises data, virtually as flat as GTFS would do. So it makes total sense that you wouldn't get the exact same result back. But the second time around, you would.
However, think I would find this system a bit more understandable if the service journeys would refer to a specific version of the pattern which in turn refers to a specific version of the route which in turn refers to the specific version of the line which has the validity period. As
I don't see that in the feed I have at hand. Can you confirm that it _should_ be as I described?
Allow me one more question: shouldn't there also be two versions of the pattern and the route? Basically one for each validity period?
Yes - as I already mentioned, there are two versions of ServiceJourneyPattern AND there should be two versions of Route.
Can you help me understand how consumers are expected to apply the filtering logic?
If I want to figure out a service journey's operating days I have to take its operating days then go up the object tree to pattern -> route -> line and then apply the line's validity period to the operating days and remove everything that falls outside of that?
Or can I simply drop at import time all service journeys whose line version is not valid "today". I'm assuming I would then get incorrect service journeys/patterns for the time after the currently active version has expired. But maybe not?
I inspected the full document and see that both mentioned ServiceJourneys are part of the same TimetableFrame. Thus they share they same external validity boundary and I would therefore regard the document as improper and that it should be rejected.
I understand that the producing system uses Line-version as the top node, this is of course fine, but different from the bottom-up approach of the EPIP NeTEx-structure. This means that some transformation work is needed to get a usable export. One way would be to use adapted validity-bitpatterns for each ServiceJourney.
An even simpler way of bringing the export in line with EPIP would be to use multiple TimetableFrames. One TimetableFrame per Line-version.
All ServiceJourneys that belong to a certain Line-version would then be placed into a Line-version specific TimetableFrame having the same validity-condition as the Line version that the ServiceJourneys belong to. In this case it would be OK to use the same validity-bitpatterns as EPIP states that the Frames validity takes precedence.
We should also add more text in EPIP.
Aspects that many of us thought where obvious are not actually stated explicitly in the EPIP-document. We lack the following clarification:
...any referenced element, that is not optional, must be valid on all dates that the referring element is valid. This applies also across chains of references.
I would absolutely love it if this could be clarified. What is the process of amending the spec, either EPIP or NeTEx itself?
I think it would be great if we could come up with an example of how versioning should work here: https://github.com/NeTEx-CEN/NeTEx/blob/master/examples/standards/epip/epip_common_profile.xml
This file, for example, has no versioned entity at all, just versioned frames, AFAICS.
I'm happy to contribute to that. Maybe Mentz could give input about their line versioning feature, too.
<ServiceJourney id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-2-25260:T0:" version="2">
refers to
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a200505:" version="2"></ServiceJourneyPatternRef>
which refers to
<RouteRef ref="it:apb:Route:3-250-S-25a-2-5/H:" version="any"></RouteRef>
<Route id="it:apb:Route:3-250-S-25a-2-5/H:" version="any">
<LineRef ref="it:apb:Line:03250S.25a:" version="2"></LineRef>
and
<ServiceJourney id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-3-25260:T0:" version="3">
refers to
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a300505:" version="3"></ServiceJourneyPatternRef>
<RouteRef ref="it:apb:Route:3-250-S-25a-3-2/H:" version="any"></RouteRef>
<Route id="it:apb:Route:3-250-S-25a-3-2/H:" version="any">
<LineRef ref="it:apb:Line:03250S.25a:" version="3"></LineRef>
So the ServiceJourney with version 2 ultimately refers to the Line with version 2, and the ServiceJourney with version 3 ultimately refers to the Line with version 3. Am I overlooking something?
The easiest way to interpret the validity is by starting from LINE, a version is valid until the next version starts. You can have temporary versions, e.g. you can have a basic line version from 17.2.25 to 31.12.25 and a "detour" version from 01.05.25 to 03.05.25.
@all: OK, this may be somewhat counter-intuitive. I still do not think it is illegal, but I do not claim to be the Pope of NeTEx.
What we can do is we can generate the day type validity so that it describes the validity of the ServiceJourney with respect to the containing timetable frame, so you would not have to interpret the validity by overlaying the validity of the line version.
Introducing versioned TimetableFrames sounds legitimate too, but that would be a breaking change and we would have to confer with all users before.
`version="250210170454"`
I.e. its not actual version tracking, just a fill. When the data source doesn't provide actual version tracking, should I recommend they use version="any" or version="0"?
Unless the version has a clear meaning (and is preferably incremental, so missing versions can be quickly spotted), I would use version=0, and perhaps changed="2025-02-17T17:04:54"
Having referenced object version "1 " and referencing object version "any" will produce a schema validation error - and even if it did not, there would be a some interpretation necessary to state which version of the referenced object you mean.
The schema validation error is caused by the inability of XML Schema to do something "magic" with any. Personally, I don't think any should be used if the producing system just knows what the exact version of the object is anyway.
For example, version=2 should not be valid unless there is a version=1 for the same ID.
But regardless... if the version "3495845" or "250210170454", or "any", it essentially means version="idontknow" or version="idontcare".
And of course, version is required, so everyone will put something in there.
This is the part that which threw me. I thought that the version only applies to the entity itself and cannot be used as a global selector for other entities "up the object tree" and if you want SJv2 to have a different validity from SJv3 then you need to point them to JPv2/3, Routev2/3 and eventually Linev2/3.
I'm too much of a newcomer to make definitive claims about whether this statement is true or false. Others in this thread with more experience have commented though.
One thing we can agree on though is that the spec could be a lot clearer about this.
This or the separate TimetableFrame is exactly how I would have expected it.
I looked the feed today and I am scratching my head a bit.
Before I investigate more, I would like to ask