S9 Skew investigation - your help needed!

[ozij]

Not sure if this is relevant:

ResScan 3.12, Firmware 0602

Detailed graphs screen has an upper part, for Navigation, and lower part for Details:
I use the navigation (top) part for longer time periods and navigating. I use only bottom part (Details) to focus on details.
I often change the number and type of graphs I want to see in each panel, using the "graphing options panel" (leftmost tab after the machine's serial number == ReScan 3.12 manual pages 26 and thereabouts).

Each of the details part charts has its own red line down the center.
For some charts, the red line comes up misaligned.
I go to the above mentioned "graphing options panel" selection tab.
In the graph selection for the detailed charts I deselect the misalingned graph and immediatly reselect it.

All center lines realign. I haven't noticed other skews, I've only bee running the machine for 10 nights.

Edited to add: I get the impression that this realignment is a necessary kind of screen refresh -- perhaps more necessary when the software has more detailed graphs to handle.
O.

[Blrfl]

Just my 2c: I just installed Resscan on a new laptop and downloaded the data from my SD card. The skew was present, which I think implicates the firmware more than Resscan.

Windows is full of things that don't happen consistently between installations, and many vendors have customized versions of the OS that set things differently. If this is a ResScan problem, it's entirely possible that the same condition is present on your new laptop as on your other system.

Idamtnboy examined the raw data from his card without ResScan (discussed in viewtopic/t57224/S9-data-skew--its-not- ... -card.html) and said he didn't find the skew in it, at least for the one event he studied. If true, that's good news because ResMed can correct ResScan without having to recertify the S9.

I'm relatively new to all of this, having just started therapy last Thursday, but I've started taking some steps toward doing some of the same things as idamtnboy in terms of converting the data from EDF and putting it to use without ResScan. We're fortunate that ResMed is using a documented data format and not something proprietary, because it can be used to determine conclusively whether the problem is occurring in the machine or in ResScan. Just looking at what comes out of ResScan isn't helpful, because the data will have been mangled twice: once by the machine between the patient and the SD card and again by ResScan between the SD card and the charts it generates.

I've already picked apart enough data from my machine to determine that it wouldn't be difficult to write a program to go through an entire card's worth and identify discrepancies between the events and flow data. If the skew is present in the raw data and I can reliably identify it, I should also be able to generate corrected data that you could put onto a SD card and load into ResScan as if it came directly off the machine. (There is one technical hurdle to that, which I'm looking into.)

If there's enough interest and enough people are willing to send me a card or two, I'll develop something so we can do a large-scale study of the raw data. I don't promise that this will happen quickly, but I'm interested enough in the problem to spend some time on it.

--Mark

[idamtnboy]

Windows is full of things that don't happen consistently between installations, and many vendors have customized versions of the OS that set things differently. If this is a ResScan problem, it's entirely possible that the same condition is present on your new laptop as on your other system.
--Mark

I was thinking pretty much the same thing myself.

We're fortunate that ResMed is using a documented data format and not something proprietary, because it can be used to determine conclusively whether the problem is occurring in the machine or in ResScan. Just looking at what comes out of ResScan isn't helpful, because the data will have been mangled twice: once by the machine between the patient and the SD card and again by ResScan between the SD card and the charts it generates.

I hesitate to say the data is mangled twice. The original data file is a straight forward record of what the machine reads from its transducers and clock, except for calculating what is and isn't an event. Of course the transducer outputs are massaged from the raw electrical signals to create an understandable standard number, but that type of technology is decades old so I wouldn't give that part of it a second thought, really. The data is mangled by Resscan though. The various data files on the card are processed and then stored in another data file altogether with the extension RLK. RLK files are probably encrypted, or at the very least some sort of number shifting scheme applied to the data, as they are not in an understandable format.

I've already picked apart enough data from my machine to determine that it wouldn't be difficult to write a program to go through an entire card's worth and identify discrepancies between the events and flow data. If the skew is present in the raw data and I can reliably identify it, I should also be able to generate corrected data that you could put onto a SD card and load into ResScan as if it came directly off the machine. (There is one technical hurdle to that, which I'm looking into.)

Is the technical hurdle the CRC file? I downloaded a program called EasyHash 1.5 which has a bundle of algorithms for generating CRCs. None of them would generate a CRC that matched the S9 generated CRC. Resscan generates a CRC of the file on the card and compares it to the one from the S9. No match, no download. Change even one byte in the S9 data file and it won't download into Resscan.

If there's enough interest and enough people are willing to send me a card or two, I'll develop something so we can do a large-scale study of the raw data. I don't promise that this will happen quickly, but I'm interested enough in the problem to spend some time on it.

All you need is the set of files from a card. You can then copy them to an SD card. Resscan will read the card as long as the data is stored on the card correctly.

[idamtnboy]

IT HAS TO BE RESSCAN!!!
I discovered that Resscan includes several sample data files for various types of machines. I imported the S9 example as another patient and then looked at the graphs. Guess what? Resmed's own detail data file has a ~40 minute skew when I look at it with the nav window set to 8 hrs and the detail window set to 5 minutes, the same skew I see on my data. Now, you really don't think Resmed would include defective data in the Resscan installation, do you?

[Blrfl]

I hesitate to say the data is mangled twice.

I was being cute (or making a weak attempt at it, anyway). There are two places where there's potential for mangling. I don't dismiss anything, even the obvious stuff that should be right, because that's often where the problems lie.

Is the technical hurdle the CRC file? I downloaded a program called EasyHash 1.5 which has a bundle of algorithms for generating CRCs. None of them would generate a CRC that matched the S9 generated CRC.

Yes, it is. (And I did get your PM, BTW.) My thought is that CRC might just be on the data or starting at some offset in the file that isn't zero. I just haven't had time to strip the headers out and try it since I thought of it.

All you need is the set of files from a card. You can then copy them to an SD card. Resscan will read the card as long as the data is stored on the card correctly.

If we really think it's the raw data, I wasn't even going to bother with ResScan to study the problem. I'd take a card's worth of data from 50 different machines, dump it into a database and look at every event and see how it matches up with the corresponding flow data.

--Mark

[DreamDiver]

Actual therapy trumps quirky data.

You worked on the data skew issue a lot last spring. What's your take on what I present in my thread today about S9 data being a Resscan problem, not a machine or card problem?

I discovered that Resscan includes several sample data files for various types of machines. I imported the S9 example as another patient and then looked at the graphs. Guess what? Resmed's own detail data file, the rlk files in the patient folder, has a ~40 minute skew when I look at it with the nav window set to 8 hrs and the detail window set to 5 minutes, the same skew I see on my data. Now, you really don't think Resmed would include defective data in the Resscan installation, do you?

Also, if you look at flow and events in the same window, either nav or detail, you will see they are lined up perfectly. The problem is how Resscan applies the timeline to the windows.

They did in fact leave a skew in the example patient data. Remember, they're using the same hardware we are to create their example data. It only makes sense the data would show the skew, even if unintentionally. Falsifying example data might be considered scientifically unethical. Science is all about exposing the warts. You're just seeing that in practice. I'd be more likely to think twice about Resmed's ethical standards if the skew didn't show in the example data. I applaud ResMed for not giving us adulterated example patient data.

Proof of Skew Root Cause:
The data is stored on the SD card in a widely-renowned open-source format (.edf, amongst others) These files are readable in an open-source program like EDFBrowser. Once you figure out how to display the data, it's easy to see that the skew exists in the original format before it even gets imported into ResScan.

ResScan manipulates the data as it pulls it from the SD card and places it in .rlk format (again, amongst others) in the personal data folders within the ResScan program folder on a windows machine. Edited data is not first-hand data.

EDF Browser views the data unchanged, directly from the original document (or a duplicate) in which the data was actually recorded. The skew is visible in the first-hand data just as precisely as it is seen in ResScan's edited version. Since we know EDFBrowser is only viewing and not manipulating the data into another file type, we can safely assume the data error is with the S9 firmware - not with ResScan. You can get EDFBrowser for Linux and see the same results, so this glitch is independent of the operating system upon which it is viewed.

Adapting ResScan to 'fix' the problem as an end-user Solution:
Consider the possibility that they could alter ResScan to look for and adapt the data in an attempt to fix the skew at the second-hand data level. The product of that manipulation then becomes third-hand data. It gets worse because the manipulation involved is a like leap-year calendar fix on an accelerated scale. One 'little' adjustment algorithm may introduce the need for increasingly-more-subtle adjustments over longer periods in an already-questionable adjustment algorithm. From a scientific and medical standpoint, the resulting output will not garner as much trust as first-hand or second-hand data. Over a six-month period, the altered skew would probably show additional anomalies. ResMed will know this if they have attempted to fix it. Either that or they'll find it out when they do attempt it. A poor xerox copy of a poor xerox copy of a xerox copy... Meh.

End Result...
This is a firmware issue*. It will not be readily resolved with software. The only fix is a firmware fix. Unless something in the firmware is found medically unsafe, the machine firmware will remain unchanged for all future production of the current S9 family. A firmware upgrade for existing S9's is very unlikely because the flow-generation part of the firmware is truly an awesome thing -- and entirely independent of the data-recording part of the firmware. Count on it.

Format the SD card every two weeks to avoid gross skew. This is the best we can do as end users.

*While it's only been briefly discussed on this forum there is a remote possibility that it could be a hardware issue. There may be one or more conflicts between integrated circuit chips on the flow generator's circuit board that are hard-wired into the recording end of the system. If that's the case, you can forget even a firmware fix. That said, this consideration is extremely remote. I've only seen something like this once: Some iMacs have a faulty random-number generator due to a hardware issue that makes them vulnerable to all sorts of problems for software developers who aren't aware of the problem.

[idamtnboy]

Once you figure out how to display the data, it's easy to see that the skew exists in the original format before it even gets imported into ResScan.

OK, it's time for a mea culpa. I failed to go back to your original post to make sure I was seeing the same problem you described. I'm not. What I am seeing is a timeline disconnect between the nav screen and detail screen that shows up regardless of what data set I'm looking at. The problem you discovered, and as shown in your original posted screen shots, is a time shift clearly visible in both screens between an event and the corresponding flow. I have Resscan 3.12. In my case there is no time difference between the event and flow when viewed in the same window.

So, it appears we have two issues, time difference in the original data as you have so well described and discussed, and a display problem with Resscan that I'm seeing.

Again, I apologize for confusing the problem I'm seeing with the problem you made known.

[DreamDiver]

Again, I apologize for confusing the problem I'm seeing with the problem you made known.

No Worries.
However... Now I'm confused.
Have you posted images of the new issue in 3.12?
I'd like to see what you're talking about. I'm still using 3.11.
Since it is a new issue, if you haven't already, it might be worth posting a new thread about it.

[idamtnboy]

No Worries.
However... Now I'm confused.
Have you posted images of the new issue in 3.12?
I'd like to see what you're talking about. I'm still using 3.11.
Since it is a new issue, if you haven't already, it might be worth posting a new thread about it.

Yesterday evening, here. viewtopic.php?f=1&t=57224&p=537959#p537959 S9 data skew - it's not the CPAP nor card

[DreamDiver]

No Worries.
However... Now I'm confused.
Have you posted images of the new issue in 3.12?
I'd like to see what you're talking about. I'm still using 3.11.
Since it is a new issue, if you haven't already, it might be worth posting a new thread about it.

Yesterday evening, here. viewtopic.php?f=1&t=57224&p=537959#p537959 S9 data skew - it's not the CPAP nor card

Please see my reply there.

[ozij]

Edit:
Some interesting data about the skew bug...
Regardless of data transfer best practices, a skew between flow and detailed data keeps appearing in my graphs.

• There is always a minimum of one second in offset between flow data and detailed data in the graphing detail pane.
• The skew first starts out showing the beginning and end of the flow offset by a number of seconds from all other detailed data.
• That number of seconds seems to almost double on a daily basis after a certain number of mask events.
• The offset is between flow (hi-res) data and detailed data. In contrast, detailed data seem to stay aligned with each other. However event data show a peculiar difference, possibly because they are point data rather than line data.
• At first, event data continues to line up correctly with flow data, even though line graphs of detailed data continue to skew wider and wider at the beginning and end of each mask event. After an unknown threshold of mask events, the skew begins to show up in the apnea/hypopnea event data graph as a massive skew between where it appears in flow and where it appears in event graphing.
• Skew lengths are not always predictable, other than that they grossly tend to get larger. For instance, sometimes the skew will be four seconds at the beginning of a mask event and six seconds at the end of the same mask event. Sometimes the skew will be opposite what it was at the beginning of the mask event.
• Here's one item that is predictable: Flow data must always start and stop on the same even numeral.
  • March 18th was the only day I found where start/stop times were actually odd, yet they remain the same numeral for start and stop:
  • March 18th Flow Data - Mask event start and stop times:
    • Mask event 1 - started at 18:55:55 and ended at 19:20:55
    • Mask event 2 - started at 00:48:51 and ended at 05:40:51
    • Mask event 3 - started at 06:11:55 and ended at 06:46:55
• Another item that is also always predictable: Detailed data must always start and stop on an even numeral, the last of which is always 2 seconds less than the first:
  • March 18th Detailed Data - Mask event start and stop times:
    • Mask event 1 - started at 18:56:06 and ended at 19:21:04 | 06 - 04=2
    • Mask event 2 - started at 00:49:38 and ended at 05:40:36 | 38 - 36=2
    • Mask event 3 - started at 06:12:46 and ended at 06:46:44 | 46 - 44=2
• March 18th - Flow minus FL start and stop time differences:
  • Event
    Mask event 1
    F - FL start | 11 sec
    F - FL stop | 9 sec
    
    Mask event 2
    F - FL start | 47 sec
    F - FL stop | -15 sec
    
    Mask event 3
    F - FL start | 51 sec
    F - FL stop | -11 sec
• March 18th is also the day when the skew shows up in the events.
  It's particularly interesting in that the first mask event shows no skew in event graphing, but the second mask event does.
  • March 18th Event graphing
    • Mask event 1 - no visible skew between flow and event graphs
    • Mask event 2 - about 11 seconds visible skew between flow and event graphs
    • Mask event 3 - no apneas - n/a
• Regardless of whether I lock the SD card when transferring the data to ResScan 3.10, the skew continues to grow.
• For some people, the skew does not appear to be happening. They may not be looking as closely, so I cannot say.
• The only way for me to temporarily reset the skew is to wipe the card and start again. Since the card is locked during data entry, forum members have (perhaps erroneously) concluded that the bug resides in the firmware of the S9.

First Mask Event - No Event Graph Skew


Same Day, Next Mask Event - Visible Skew

After 14 nights of use, I've started seeing this skew too, on 3.12. The flow data starts earlier, and end later -- and the skew appears relative to minute ventilation and respiratory rate as well - it makes sense in relation to the latter two, since (breaths) can be registered immediately, whereas respiratory rate and minute ventilation both need to collect data and make calculations before statring to record teh first data poing. That, however, does not explain the Event skew, which is also there.

Point to ponder: the SD was 85% fragmented this morning.

[dave21]

ResMed does NOT acknowledge the problem in my experience... partially because their "customers" are not the end users and thus NOT an important part of the sales chain.

I managed to get ResMed to acknowledge the problem when I had to return my S9 and H5i due to another fault that nobody else seems to have hit here in the forums. Resmed said it was fixed in the SX474-0602 although I'm still seeing the skew, in-fact after one month the skew has now reached 30 seconds.

I just learn to live with the skew now, the machine and the detailed stats it gives for me far outweighs the skew problem. The skew problem is annoying and I hope ResMed eventually get their head around it.

Thanks
Dave

[ozij]

My firmware is SX474-0602. ResScan 3.12

[idamtnboy]

My firmware is SX474-0602. ResScan 3.12

Same with mine.

My observation is skew is caused by S9 data recording problems.

See my comments posted last night here viewtopic.php?f=1&t=57224&p=539392#p539392