The data provider is wrong and doesn't comply to EPIP.
Also this is not the way versions are intended to work. Well, they are poorly defined and even the expert group doesn't agree how to use them really.

version could be used:
* referencing a specific version
* telling the data production event
The usage of any usually indicates that you don't do it. The xsd checks can't handle "any" correctly. If one starts using any then one should not really do interesting stuff.

But one thing is certain:
The valididy of each object is defined by its validity and not some messed up idea of working through the object tree and trying to find out, what might change it. So, certainly the Line is not used to determine the operating days of a service journey.

Hi all, we (MENTZ) are the creators of this dataset, so I think I can shed some light on this. In our system DIVA, you can use line versions for modelling of service changes, e.g. diversions, extra timetables for certain events etc. This existed before NeTEx and is used by many of our customers. We are also not the only scheduling system with line versions. We think that NeTEx allows for using versioning in this way. Even though this is not explicitly mentioned in the EPIP profile, the CEN document says this:

In our system, versioning can affect properties of the LINE as such, but mostly service patterns and journey are affected. 
So you should be able to go from ServiceJourney to ServiceJourneyPattern to Route and Line or the other way around. If there is an inconsistent reference between the versions of these objects we would consider this a bug.

So my first question is why does the NeTEx feed contain data that must immediately be filtered out?  Is there some sort of external condition that is used to decide what is to be retained and what is to be filtered out; then, if that external condition changes, a different timetable should apply?

Remember that you have a data hierarchy.  Don't select the Service Journey by going up the data tree to the Line, go the other way round.  Start with the Line.  Does it meet your external selection criteria?  If so, select all the Journey Patterns that reference your selected Line(s).  Then select all the Service Journeys that reference the selected Journey Patterns. 

Mike something else happens here. What Mentz is doing, and I don't even know if it was actually being prevented from the EPIP profile (but certainly from the spirit of it), is introducing the ability to update key concepts during the lifecycle of the publication. For example when a Line name is changed. But the example presented by Leonard shows actually nothing changes. And that is for me a red flag.

Mike Yes correct, it is much more intuitive to start from the LINE. A dataset usually contains the planned services and journeys for the timetable period; if you only want to know what operates next week, you only look at the versions valid next week.

Stefan : We had the discussion about usage of versions countless times, and the "spirit" is not really an IT term. We did the same for the implementation of the Austrian NAP. This was reviewed and accepted by https://austriatech.at/, who also submitted it to data4pt.org.  Nobody complained.

The alternatives would be 
a) Tell our customers that if they use Netex they cannot use line versions - not really an option
b) Merge all of our line versions into one NeTEx version. Technically possible, but at the cost of information loss (Re-Import of data would not yield different line versions)

I would say that we lack information for concluding if the data is valid or not. We need to see the Calendar and the Validity-period of the containing VersionFrame.  I suggest posting the complete document.

However, note that

the two mentioned ServiceJourney-element have different Ids and are defined for the same DayType, so they exist in parallel if they are part of the same delivery.

Generally the data-delivery should be rejected as a whole if any of the included ServiceJourney-versions are valid on any dates, during the validity-period of the containing VersionFrame, that the referenced Line is not valid. In our world it is a difference if you refer to a Line by Id or by Id + version.
In the first case it is ok as long as any version of the Line is valid on all the dates found by combining DayType with the Calender during the validity-period of the VersionFrame . In the later case the referred version of the Line must cover all the dates.

Wilfried Data4PT did not have the expertise let alone the correct tools to do a sound analysis. The data I checked, for example for Portugal or Luxemborg did not even meet the criteria of XML Schema validation.

Regarding the spirit: none the EPIP-examples have been using the VDV-like ValidBetween. Hence Line-Version is invented by Mentz. A construct which is used in a Mentz system not 'simple for a generic data user' the EPIP-profile was targeted for. Yes, EPIP-materialises data, virtually as flat as GTFS would do. So it makes total sense that you wouldn't get the exact same result back. But the second time around, you would.

Wilfried I understand that customers want to have variations of a specific service at specific time period.

However, think I would find this system a bit more understandable if the service journeys would refer to a specific version of the pattern which in turn refers to a specific version of the route which in turn refers to the specific version of the line which has the validity period. As Ulf  has pointed out in  such a scenario the service journeys should have the same id, shouldn't they?

I don't see that in the feed I have at hand. Can you confirm that it _should_ be as I described?

Ulf I'm not 100% certain I am allowed to post the complete file but I can ask.

Wilfried Do you know?

Please ask STA Bolzano - I do not think they will object. From what I see in the fragments you posted, ServiceJourney does refer to a specific version of ServiceJourneyPattern, but ServiceJourneyPattern refers to a Route where the version has been put into the id string. While that will technically work, it is not nice. If you want it changed, please contact STA - we cannot change it just on the basis of this discussion.
 

Wilfried Thanks. I will raise this with STA.

Allow me one more question: shouldn't there also be two versions of the pattern and the route? Basically one for each validity period?

Not if "any" is used as version-attribute in the ref.

Leonard  
Yes - as I already mentioned, there are  two versions of ServiceJourneyPattern  AND there should be two versions of Route.

Wilfried I'm sorry to say, but there is only a single version of ServiceJourneyPattern but two ServiceJourneys with different IDs and different versions (but also encoded in the ID). I'm assuming that's what you meant to say.

Can you help me understand how consumers are expected to apply the filtering logic?
If I want to figure out a service journey's operating days I have to take its operating days then go up the object tree to pattern -> route -> line and then apply the line's validity period to the operating days and remove everything that falls outside of that?
Or can I simply drop at import time all service journeys whose line version is not valid "today". I'm assuming I would then get incorrect service journeys/patterns for the time after the currently active version has expired.  But maybe not?

Ulf I have received the permission to post the full feed: https://leonard.io/noi/sta.netex.zip

Hi!
I inspected the full document and see that both mentioned ServiceJourneys are part of the same TimetableFrame. Thus they share they same external validity boundary and I would therefore regard the document as improper and that it should be rejected. 

I understand that the producing system uses Line-version as the top node, this is of course fine, but different from the bottom-up approach of the EPIP NeTEx-structure. This means that some transformation work is needed to get a usable export. One way would be to use adapted validity-bitpatterns for each ServiceJourney.
 
An even simpler way of bringing the export in line with EPIP would be to use multiple TimetableFrames. One TimetableFrame per Line-version. 
All ServiceJourneys that belong to a certain Line-version would then be placed into a Line-version specific TimetableFrame having the same validity-condition as the Line version that the ServiceJourneys belong to.  In this case it would be OK to use the same validity-bitpatterns as EPIP states that the Frames validity takes precedence.

We should also add more text in EPIP.

Aspects that many of us thought where obvious are not actually stated explicitly in the EPIP-document. We lack the following clarification: 

...any referenced element, that is not optional,  must be valid on all dates that the referring element is valid. This applies also across chains of references.

Thanks for looking at the feed, Ulf .

I would absolutely love it if this could be clarified. What is the process of amending the spec, either EPIP or NeTEx itself?

I think it would be great if we could come up with an example of how versioning should work here: https://github.com/NeTEx-CEN/NeTEx/blob/master/examples/standards/epip/epip_common_profile.xml

This file, for example, has no versioned entity at all, just versioned frames, AFAICS.

I'm happy to contribute to that. Maybe Mentz could give input about their line versioning feature, too.

The better answer would be that EPIP should not do constraining versions, but only exports with the current version. I am for the Norwegian interpretation of versions that an object may have its own version attribute, as long as the references are explicitly made.

I would thoroughly agree with the suggestion of using multiple TimetableFrames - one per Line-Version.  That provides a clear division of the data, and is in line with the Norwegian interpretation, where each distinct Line (aka Line Version) has its own paired TimetableFrame.

Leonard : I still do not see where the chain of reference from ServiceJourney to Line is broken:
<ServiceJourney id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-2-25260:T0:" version="2">
refers to
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a200505:" version="2"></ServiceJourneyPatternRef>
which refers to 
<RouteRef ref="it:apb:Route:3-250-S-25a-2-5/H:" version="any"></RouteRef>
<Route id="it:apb:Route:3-250-S-25a-2-5/H:" version="any">
<LineRef ref="it:apb:Line:03250S.25a:" version="2"></LineRef>
and
<ServiceJourney id="it:apb:ServiceJourney:032508S-SAD_Bahn-4-3-25260:T0:" version="3">
refers to
<ServiceJourneyPatternRef ref="it:apb:ServiceJourneyPattern:03250S.25a300505:" version="3"></ServiceJourneyPatternRef>
<RouteRef ref="it:apb:Route:3-250-S-25a-3-2/H:" version="any"></RouteRef>
<Route id="it:apb:Route:3-250-S-25a-3-2/H:" version="any">
<LineRef ref="it:apb:Line:03250S.25a:" version="3"></LineRef>

So the ServiceJourney with version 2 ultimately refers to the Line with version 2, and the ServiceJourney with version 3 ultimately refers to the Line with version 3. Am I overlooking something? 

The easiest way to interpret the validity is by starting from LINE, a version is valid until the next version starts. You can have temporary versions, e.g. you can have a basic line version from 17.2.25 to 31.12.25 and a "detour" version from 01.05.25 to 03.05.25.

@all: OK, this may be somewhat counter-intuitive. I still do not think it is illegal, but I do not claim to be the Pope of NeTEx. 
What we can do is we can generate the day type validity so that it describes the validity of the ServiceJourney with respect to the containing timetable frame, so you would not have to interpret the validity by overlaying the validity of the line version. 
Introducing versioned TimetableFrames sounds legitimate too, but that would be a breaking change and we would have to confer with all users before. 

Sidestep but on topic. I've gotten a dataset with this:
`version="250210170454"`

I.e. its not actual version tracking, just a fill. When the data source doesn't provide actual version tracking, should I recommend they use version="any" or version="0"?

I consider an object with version="any" a very bad practise (opposed to a reference with version="any"). So I would suggest do something like 202502171622 as the date time. 

So the version="250210170454" is simply saying this was the value as at 10-Feb-2025 17:04:54.  

Unless the version has a clear meaning (and is preferably incremental, so missing versions can be quickly spotted), I would use version=0, and perhaps changed="2025-02-17T17:04:54"

Stefan If an object with version="any" is bad practice, then it should not have been put into the NeTEx XML example in the first place. We assumed that you can use "any" if you do not have versions for the object in the source system. 
Having referenced object version "1 " and referencing object version "any" will produce a schema validation error - and even if it did not, there would be a some interpretation necessary to state which version of the referenced object you mean.

I totally agree. But even more stupid examples exists too. In the documentation "any" is suggested that a reference can refer to "any" object.

The schema validation error is caused by the inability of XML Schema to do something "magic" with any. Personally, I don't think any should be used if the producing system just knows what the exact version of the object is anyway.

I would say, use versions for the correct purpose, or not at all.
For example, version=2 should not be valid unless there is a version=1 for the same ID.

But regardless... if the version "3495845" or "250210170454", or "any", it essentially means version="idontknow" or version="idontcare".

And of course, version is required, so everyone will put something in there.

> So the ServiceJourney with version 2 ultimately refers to the Line with version 2, and the ServiceJourney with version 3 ultimately refers to the Line with version 3. Am I overlooking something? 

This is the part that which threw me. I thought that the version only applies to the entity itself and cannot be used as a global selector for other entities "up the object tree" and if you want SJv2 to have a different validity from SJv3 then you need to point them to JPv2/3, Routev2/3 and eventually Linev2/3.

I'm too much of a newcomer to make definitive claims about whether this statement is true or false. Others in this thread with more experience have commented though.

One thing we can agree on though is that the spec could be a lot clearer about this.

We are exploring with the agency whether we can export only the "current" version from Mentz. Not sure if the journeys outside the current validity will be correct though.

> What we can do is we can generate the day type validity so that it describes the validity of the ServiceJourney with respect to the containing timetable frame, so you would not have to interpret the validity by overlaying the validity of the line version.

This or the separate TimetableFrame is exactly how I would have expected it.

It's a year later and am asked to review the situation with versions in South Tyrol.

I looked the feed today and I am scratching my head a bit.

Before I investigate more, I would like to ask Wilfried if Mentz has implemented any of the changes that we have discussed here.

Leonard  we do not change our implementation just based on NeTEx basecamp discussions. If you want the implementation changed, please discuss it with our customer (STA). Hint: For the VDV 462 profile we do have an option to resolve line version validities to ServiceJourney day type validities, which makes it easier to interpret, as you have to look at one location for validities only.