Philips Respironics System One REMstar Pro file format

[CPAPito]

That is very impressive!!

I had managed to look at the graph on my audio editing program, but had no tools to analyse it.

Already ordered EV 2.0 - will be interesting to compare the results. If I can be of any help, do let me know!

Thanks and best,

C

I've made a small web application that will let you visualize data on your .005 files. In order to use, you'll need to be able to retrieve your 005 files (as explained earlier in this topic) from your SD card. There are some examples from my own therapy sessions there, as well.

The application will plot the data and make a simple effort to detect "Events". Please note these events are not defined by the CPAP machine nor do they necessarily have any physiological or medical significance.

I've only tested it with data from my own machine, so if you try it and notice any problems, please let me know (here or via the email link on the application).

Lastly, I'm sensitive to the privacy concerns you may have about submitting your CPAP data to a website. Please read the short privacy statement on the website before you submit any data.

Visit the PRS1 Data Viewer at http://www.hoolehan.com/onkor/

I hope you find this useful!

[milestogo]

I've made a small web application that will let you visualize data on your .005 files. In order to use, you'll need to be able to retrieve your 005 files (as explained earlier in this topic) from your SD card. There are some examples from my own therapy sessions there, as well.

The application will plot the data and make a simple effort to detect "Events". Please note these events are not defined by the CPAP machine nor do they necessarily have any physiological or medical significance.

I've only tested it with data from my own machine, so if you try it and notice any problems, please let me know (here or via the email link on the application).

Lastly, I'm sensitive to the privacy concerns you may have about submitting your CPAP data to a website. Please read the short privacy statement on the website before you submit any data.

Visit the PRS1 Data Viewer at http://www.hoolehan.com/onkor/

I hope you find this useful!

Wow! this is awesome! I just hit upon this thread looking for the very same info and would just like to report my findings.

I looked at the .005 file through hexedit and it seems the first 12 bytes are UNIX style data format. The next 1514 bytes are "data". then the timestamp (12 bytes) is repeated again. This is how far I've gotten ...

I'm not sure if the web app is using an 8 byte or 12 byte date field?. Are my observations correct? it seems from your observations the first 8 bytes are being taken as the timestamp.

I'm wondering if you or anyone has had any success deciphering the "data fields" within the data section? are they 16/32 bits long? Awesome job making the webapp! this will be very useful as new information becomes available!

PS. I wish respironics would publish the data format and allow users to build a better reporting application than the one they offer.

[milestogo]

To continue that train of thought, couple more thoughts on the file format before it "slips my mind" as it were:

NOTE: this is pure speculation at this point (and based on my assumption that the date record is 12 bytes not 24 as mentioned by @rdrd and @cpapdork :

12 bytes date/time stamp
1500 bytes for 300 seconds of sampling (5 bytes per second). Could it be 1 "base variable" per byte i.e., 5 vars collected per second?
14 bytes could be "other data" ? maybe averages of the last 300 seconds?

The reasoning behind the "base variable" idea is that they're probably collecting 5 main "counters" or "variables" per second and then calculate everything else based on those numbers.

I haven't done any more investigations in terms of correlating the size of files, number of timestamps in a given file yet but will report back any other interesting facts or just ideas to throw around.

It would be really cool if we could decode this format and make it useful for general users. I have half a mind to get myself a scope and a data probe/sniffer to see what the heck the MCU is sending down the pipe but my microcontroller-fu is not that strong. (Plus I don't want to brick my new respironics machine...)

[milestogo]

...
12 bytes date/time stamp
1500 bytes for 300 seconds of sampling (5 bytes per second). Could it be 1 "base variable" per byte i.e., 5 vars collected per second?
14 bytes could be "other data" ? maybe averages of the last 300 seconds?
...

I think I might be out to lunch on the "base var" hypothesis. It seems these are just pressure readings. I do think these are being done 5 times a second (1500 distinct samples over each 5 minute (300 second) period). I did some more analysis of just one 300 second interval's data and this is what I get.

How I got these graphs:
1. load a ".005" file into hexedit.
2. Select the first 12+1514 bytes, copy, paste into a new hexedit file.
3. Strip out the first 12 bytes (datetime stamp) of the hexedit file, and save as raw.txt (or what have you)
4. import raw.txt into audacity with the following import settings:
-- signed 8bit PCM
-- no endianness
-- 1 channel mono
-- start offset = 0 bytes
-- sample rate 100 hz
5. view the resulting file and zoom in.

After doing all this, this is what I get.


This leads me to believe that the system basically just collects pressure readings 5 times a second for 300 seconds straight, writes a checksum or something in the last 14 bytes and starts a new "record" with a new timestamp. I could of course be wrong.... ... but it seems that way to me right now.

The selected area is about 20 seconds of real elapsed time and there are about 100 samples in there. There seem to be about 4 to 5 breaths ever 100 readings which about jives with a quick test I did timing my breaths.

Now, what remains to be seen is what kind of sample data each byte contains? Could it be just a straight forward millibars or something? since the sensor chips probably record in metric? Any insights into this would be much appreciated.

cheers.

[cpapdork]

milestogo:

My findings so far

The overall structure of the 005 is a repeated series of blocks, where each block consists of 24 bytes of header, 1500 bytes of data payload of signed integers, followed by 2 bytes of footer.

The payload clearly measures something directly correlated with breathing, but what this precise metric is, I do not know for certain. However, my presumption is that this it is rate of airflow to (and from) the machine. But even if so, the calibration of the datapoints with actual airflow measurements is unknown.

The 24 bytes of header is not fully decoded yet. However, byte 6 [zero indexed] denotes the file type (i.e. '05' corresponding to the 005 file extension). Bytes 7-10 constitute a 4 byte unsigned integer which indicates a sequence number matching the sequence number in the filename. Bytes 12-15 are a 4 byte timestamp of resolution 1 second, which denotes the starting time of the remaining 1500 data points.

Consecutive block headers show timestamps differing by exactly 300 seconds, therefore it is assumed that the device records data samples five times per 5 second.

The 2 byte footer is not yet decoded

Bytes 6 and 12-15 in the 001/002 files share the same properties as that of the 005 files. The remaining information in these files, however, is not yet decoded.

More here: http://www.hoolehan.com/onkor/fileformats.html

[SleepyCPAP]

cpapdork and others are way ahead of me in digging into data. Thanks so much. I'll see if I can send you a waveform printout from my PRS1 Pro machine, so you can see the way the Encore software marks various events.

You were wondering about the horizontal lines showing up in the data. I know you are working with an Auto PAP, so likely different from mine, but in the EncoreViewer reports it shows certain things which are likely constant: Cflex setting (or whether Cflex is off, or set to Cflex+), pressure (remember mine is not APAP, so there is only one pressure), and humidifier setting. I don't know if those are the types of things which show up each block header. Adjusting any of those might show a change.

As a Mac user who rarely gets access to a PC to look at data, I am very grateful for your research. I rely so much on what's on the screen, and having to reset it each day. If I had my wish, Phillips Respironics would bother to do their own web-based upload/reporting for patients, or we'd crack the data ourselves so that I could at least get the Avg CA index, Avg OA Index, Avg Hypopnea Index, Avg RERA index, Avg VS Index, and Average Leak. The screen showing AHI is good, as is Periodic breathing, but I'd like the rest of the numbers. Those of you with AutoPAPs probably would want Flow as well (The "Pro" always lists zero on that one). The "Large Leak" on the screen is useless, always zero -- Avg leak would have been better.

[cpapdork]

... or we'd crack the data ourselves so that I could at least get the Avg CA index, Avg OA Index, Avg Hypopnea Index, Avg RERA index, Avg VS Index, and Average Leak.

I agree. I (and several others it happily seems!) are trying to crack the rest of the data on these file formats. One problem I had when developing the viewer that's on the web now is that I only had data from my own machine. However, the more 005 files from other CPAPs I get (a copy is saved whenever someone uses the service) the more comparisons I'll be able to make.

[cpapdork]

By running a series of short sessions with changing pressure settings, I've tentatively identified a few more bytes...

The 001 stores information about initial settings and overall therapy summary. Here's my current best guess of info contained:

• Byte 6: File type (i.e. 001)
• Bytes 7-10: Sequence Number (matches filename)
• Bytes 12-15: Timestamp
• Byte 19: Amin from clinical setup
• Byte 20: Amax from clinical setup
• Byte 23: Initial pressure?
• Bytes 38-41: Actual min or max achieved?

And now I think the 002 files records events, such as pressure changes and perhaps even clinical events (apneas, leaks, etc).

To get further info, I think I'll need some way to reliably get the machine to increase/decrease pressure during therapy at known times. Anybody have any ideas on how that could be done? Strap the machine on and hold my breath???

[milestogo]

milestogo:

My findings so far

The overall structure of the 005 is a repeated series of blocks, where each block consists of 24 bytes of header, 1500 bytes of data payload of signed integers, followed by 2 bytes of footer.

The payload clearly measures something directly correlated with breathing, but what this precise metric is, I do not know for certain. However, my presumption is that this it is rate of airflow to (and from) the machine. But even if so, the calibration of the datapoints with actual airflow measurements is unknown.

The 24 bytes of header is not fully decoded yet. However, byte 6 [zero indexed] denotes the file type (i.e. '05' corresponding to the 005 file extension). Bytes 7-10 constitute a 4 byte unsigned integer which indicates a sequence number matching the sequence number in the filename. Bytes 12-15 are a 4 byte timestamp of resolution 1 second, which denotes the starting time of the remaining 1500 data points.

Consecutive block headers show timestamps differing by exactly 300 seconds, therefore it is assumed that the device records data samples five times per 5 second.

The 2 byte footer is not yet decoded

Bytes 6 and 12-15 in the 001/002 files share the same properties as that of the 005 files. The remaining information in these files, however, is not yet decoded.

More here: http://www.hoolehan.com/onkor/fileformats.html

I think your research makes more sense. I am going to try to start a google doc with information both by yourself and @rdrd .

Currently I'm trying to look at all the dll's that were delivered with the EncoreViewr2 program. I believe the executable only runs a minimal (crippled?) version or at least a limited subset of the delivered DLL's. I'm using the dependency walker utility by Steve P. Miller ( http://www.dependencywalker.com/) however, I have not found much in terms of the functions exposed by these dll's. I do see some intriguing names in there though (especially the encore.bl.* and respironics.*.bl.* DLL's ) :

Code: Select all

          5,120 Encore.Compression.dll
        122,880 Encore.BL.RDAS.XmlSerializers.dll
         28,160 Encore.SqlServer.dll
          7,680 Encore.BL.ServiceAppMgmt.dll
         19,968 Encore.BL.EDIBulkExport.dll 
          7,168 Encore.BL.ScheduleMgmt.dll
        745,472 Encore.DAL.SQLServer.dll
         36,864 Encore.BulkImport.dll
         15,360 Encore.BL.ComplianceInformation.dll
         73,216 Encore.BL.Entity.dll
         12,288 Encore.BL.Wrappers.dll
         40,960 Encore.BL.RDAS.RTPServer.dll
         88,576 Encore.BL.Facades.dll
         73,728 Encore.BL.DataParsing.RTP.dll
         84,480 Encore.UnitTestHelper.dll
          4,096 EncoreLite.Config.dll
         49,152 Encore.Utilities.dll
         45,056 Encore.Exception.dll
         24,576 Encore.Logger.dll
         20,480 Encore.Encryption.dll
          6,656 Encore.TherapyData.DataContainer.Base.dll
         95,232 Encore.PTD.DataContainers.dll
         32,768 Encore.PSTD.DataContainers.dll
          4,096 Encore.EDI.DataContainers.Base.dll
         28,672 Encore.Validation.dll
        434,176 Encore.DataContainers.dll
         86,016 Encore.DAL.Serialization.dll
         93,696 Encore.EDI.DataContainers.dll
         61,440 Encore.DAL.Interface.dll
         25,088 EncoreLite.DataContainers.dll
         10,240 Encore.MediaController.DataContainers.dll
          8,704 Encore.ScheduleMgmt.DataContainers.dll
          5,632 Encore.ServiceAppMgmt.DataContainers.dll
         15,360 Encore.EDI.DataContainers.EDIClient.dll
         18,944 Encore.EDI.DataContainers.Import.dll
         25,088 Encore.BL.DataParsing.Common.dll
          6,656 Encore.BL.Infrastructure.dll
         81,920 EncorePro.DAL.Interface.dll
          4,608 Encore.DI.dll
         81,920 Encore.BL.Validation.Interface.dll
        114,688 Encore.BL.Validation.dll
         97,792 Encore.BL.DataParsing.SmartCard.dll   <<===============================
         49,152 Encore.MediaController.dll
         46,592 Encore.BL.Summarization.PTD.View.dll
        110,592 Encore.BL.DataParsing.UDF.dll            <<===============================
         92,160 Encore.BL.DataParsing.XmlEncorePro.dll
        118,784 Encore.BL.dll
         32,768 Encore.BL.Encoder.dll
         61,440 Encore.BL.Security.dll
         33,280 Encore.Security.dll
         86,016 Encore.BL.RDAS.dll
          9,728 Encore.BL.Interface.dll
         10,752 Encore.Auditor.dll
      1,564,672 Encore.Reporting.dll
         12,288 Encore.Reporting.BL.dll
         83,456 Encore.BL.Summarization.PTD.dll
         45,056 Encore.WinControls.dll
         10,752 EncoreLite.BL.dll
         45,056 Encore.BL.TherapyData.dll
         58,880 EncoreLite.DAL.dll
         23,040 Encore.BL.DataParsing.dll
         70,144 EncoreLite.Infrastructure.dll
      2,625,536 Encore.Reporting.TherapyDataReport.dll 
...
           49,152 Respironics.EncoreNet.BL.DataAcquisition.dll
            5,632 Respironics.EncoreNet.BL.DataAnalysis.dll
           61,440 Respironics.EncoreNet.BL.dll
           61,440 Respironics.EncoreNet.BL.EncoreCommon.dll
          294,912 Respironics.EncoreNet.BL.PatientTherapyData.dll
           94,208 Respironics.EncoreNet.BL.Security.dll
           73,728 Respironics.EncoreNet.Common.dll
          196,608 Respironics.EncoreNet.DAL.dll
            8,704 Respironics.EncoreNet.DAL.Interface.dll
           32,768 Respironics.EncoreNet.DAL.Support.dll
           20,480 Respironics.EncoreNet.Encryption.dll
           20,480 Respironics.EncoreNet.Exception.dll
           16,384 Respironics.EncoreNet.Logger.dll
           36,864 Respironics.RDAS.Common.dll
           36,864 Respironics.EncoreNet.Reporting.DAL.dll
           40,960 Respironics.RDAS.BL.dll
           36,864 Respironics.EncoreNet.Utilities.dll
           65,536 Respironics.EncoreNet.Reporting.TherapyDataReport.RDL.dll
           40,960 Respironics.RDAS.DAL.dll
           36,864 Respironics.EncoreNet.WinControls.dll
        1,941,504 Respironics.EncoreNet.Reporting.TherapyDataReport.dll
        1,294,336 Respironics.EncoreNet.Reporting.dll
            8,704 Respironics.EncoreNet.Reporting.BL.dll
           61,440 Respironics.EncoreViewer.DAL.dll

[milestogo]

Looking at the strings in the data parsing dll file (Encore.BL.DataParsing.Common.dll) , I see some interesting strings. Posted here for common research. I used procexp.exe utility to attach to the running process (via its process id in the windows task manager) ... I have tried to highlight the items that I found interesting, (full string list is pasted here: https://gist.github.com/761881 )

It seems there are indeed 24 bytes in the repeating pattern and they seem to be divided in 2 sections (mask1 is 8 bytes, mask2 is 16 bytes. what each mask refers to can be gleaned from previous messages by cpapdork and rdrd)

there seem to be three "base" type events recorded and i think everything else maybe based on these (pressure, setting, flow - events)

It seems there is some "trend" type summary also being kept in the repeating "control" sections:

Code: Select all

Encore.TherapyData.DataContainer.Base
DeviceDataSource
TransportTypes
ClassicTrendSummary
Encore.PTD.DataContainers.Statistics.SessionSummary
Enumerator
PressureChangeEvent                 <<============ base event 1
PressureChangeEventType                 <<============             ? (up? down? )    
SettingsChangeEvent                 <<============
SettingsChangeEventType
SessionStatisticData
Encore.PTD.DataContainers.Statistics
FlowEvent                 <<============ base event 2
FlowEventType                 <<============ (not sure what these might be)
CompleteSummary
VentilatorTrendSummary
SynchronyTrendSummary
TherapyEvent                 <<============ base event 3
TherapyEventType                 <<============
Math
SessionStopConditions
SessionStartConditions
DeviceSpecificTherapyData
PatientConnection
DeviceEventCategory
StateChangeEventType
<Module>
ByteEnumerator
Encore.BL.DataParsing.Common
ByteManipulator
DataBlockAsCircularQ
DataEnumerator
TherapyDeviceDataFactory
Encore.BL.DataParsing.Common.Decoders
ICircularDataBlock
IDeviceDataFormatFactory
Encore.BL.DataParsing.Common.Interfaces
IDeviceDataFormatFactoryFromSmartCard
IDeviceDataFormatFactoryFromXML
IDeviceDataFormatFromRTP
OneTimeCircularQEnumerator
realEnumerator
mask2B16b                 <<============ mask2 has 16 bytes in it
mask2B13b
mask2B12b
mask2B11b
mask2B7b
mask2B6b
mask2B5b
mask2B4b
mask2B3b
mask2B2b
mask2B1b
mask1B8b                 <<============ mask1 has 8 bytes
mask1B7b
mask1B6b
mask1B5b
mask1B4b
mask1B3b
mask1B2b
mask1B1b
<ByteOrderLittleEndian>k__BackingField                 <<============
data
startBlock                 <<============
endBlock                 <<============
dataHead                 <<============
dataTail                 <<============
enumeratorData
curIndex
endDataIndex
peekIndex
curPos
lowerBoundary
upperBoundary

[milestogo]

Some useful links regarding the algorithms used by Respironics System One machines. Really interesting are the sections on flow limitation, obstructive apnea detection etc.

• http://sleepapnea.respironics.com/techn ... rithm.aspx
• http://sleepapnea.respironics.com/techn ... ation.aspx
• http://sleepapnea.respironics.com/techn ... urces.aspx
• http://sleepapnea.respironics.com/techn ... apnea.aspx
• *Abdenbi, F., et al., "Bench Testing of Auto-Adjusting Positive Airway Pressure Devices," European Respiratory Journal 2004. Vol. 24, pp. 1-10

[cpapdork]

Ah, great.

While I know the machine itself records event data in the 002 files including (based on your DLL string dump) "Therapy Events" I haven't yet identified how they're encoded. After finishing the 001 file, that will be my next step (though I'd be pleased if someone beat me to it!).

However, ideally we could get the dataviewer to apply other researchers' algorithms as well. Perhaps let patients choose which model they prefer (I guess there's some debate as to whether patients should have all this data, but I suppose you can infer my stance on that topic.)

[cpapdork]

The software has been upgraded and the new version is available at http://www.hoolehan.com/onkor. New features include

• Storage of past data reports
• support bookmarking of your personal data page
• Upload of 001 and 002 files
• Data plot shows mode (CPAP/APAP) and pressure setting (or Amin or Amax if APAP).

Please report any errors you find!

[CruzTerri]

The software has been upgraded and the new version is available at http://www.hoolehan.com/onkor. New features include

• Storage of past data reports
• support bookmarking of your personal data page
• Upload of 001 and 002 files
• Data plot shows mode (CPAP/APAP) and pressure setting (or Amin or Amax if APAP).

Please report any errors you find!

I have a PR System One REMSTAR Pro - How do I make it see the SD Card? I have a reader and can open the card through windows.

[cpapdork]

I have a PR System One REMSTAR Pro - How do I make it see the SD Card? I have a reader and can open the card through windows.

CruzTerri, I'm afraid I actually don't know exactly how (or if) this works in Windows... I use a linux machine, and when I stick the card in my computer, it opens the card and shows the directories/files on it. Among the directories is one containing the 001/002/005 files which can be uploaded to the web service.

I don't know if this is how it works when put in a Windows computer though (I only have access to one windows computer and it doesn't have a SD card reader ).

Can someone out there give CruzTerri some advice on how to open the SD card on windows? If so and if you give your permission, I'll put those instructions on the web page, too.