ResScan UnderReporting or SmartLink OverReporting or ?

[carbonman]

I am here with a pretty clear purpose. I've been diagnosed with a medical condition. I don't like it. The fact I don't like it could contribute to some of my posts coming across as surly. I don't mean them that way - but they might. In the process of adjusting to this condition and learning more about it, I have questions.

As OSA refugees, we all came here for the same purpose.
All in various states of pain, fatigue, denial, fear and anger.
If you're pissed off about OSA, do what you got'a do.

Because of the above observation, I tend to cut newbies alot of slack.
Most cpap'ers are going to be different people after their therapy
starts to become effective.

In my initial curiosity to OSA/cpap "stuff" I caused my share of unrest.
I've been smacked down, corrected and eaten some crow.
It's all good.
It's a learning experience.......and not just the mechanics of cpap.

Ask your questions.
It's like a box of chocolates......you can't even imagine what you will get.
We will all learn from it......and not just the mechanics of cpap.

[jnk]

. . . As you did not strike me as cognitively challenged, I suspect your comment is really just more games, but am happy to field ANY sincere question or comment you may have. I further WELCOME any sincere and helpful contribution you might offer to the topic. . . .

I cannot speak for ozij. (Or anyone else, for that matter.) But she is most certainly not cognitively challenged, as you have rightly surmised. Her brilliance shines through on this forum despite the fact that American English is not, I believe, her primary day-to-day language. And if there's one thing I know about her, she does not play any psychological games when it comes to offering help to OSA patients, or the sociological structure of this forum, although she has a great sense of humor on other matters.

My post about games was more about her recognition of the "let's badmouth the industry" game that has gotten old for many here, since it rarely results in much that is constructive. You were not the instigator of that in this thread, but the thread headed that way, and ozij pointed it out in her skillful way. It must have got on her nerves. Rightfully so. Ozij takes her helpfulness seiously, so I think she just showed why she was moving on to another thread where her skills might be better utilized before she did it.

The underlying joke of my post, just so you know, especially with me and carbonman, is that I AM known to have played such games, though I am hopeful I can one day be rehabilitated. Carbonman is the designated popcorn maker for the threads that hit the hot-button issues.

Plenty of misunderstandings occur in this international forum. But for the most part, the participants rise above them, as needed, around here. My perspective of this thread is that you have asked good questions and ozij has provided good answers, but communication styles and emotional cues didn't line up particularly well before others started the whole "the industry is out to get us" thing. I am as guilty of that as anyone, as my posts in this thread show.

Bottom line is that you are welcome "here," there is always someone around to help with good questions, even if sidetracking chatter happens now and then to keep the social wheels greased along the way. This kind of forum can take thick skin. Or a persecution complex. Whichever is on sale at the time, I guess.

If any thread of yours ever gets too jacked for you, please just start another one with another question. That is never frowned upon here because this forum moves so fast that threads quickly fall down the page (or up it, depending on configuration) as varied topics come and go.

It's all good. Keep posting good questions.

I just read carbonman's post. He said it better in half the words.

[-SWS]

she does not play any psychological games when it comes to offering help to OSA patients, or the sociological structure of this forum

Doctor ozij never has in all these years, Jeff. Thank you for politely reminding us all of that and the other good things for which so many of us are grateful.

Thank you, Dr ozij.

[jnk]

she does not play any psychological games when it comes to offering help to OSA patients, or the sociological structure of this forum

Doctor ozij never has in all these years, Jeff. Thank you for politely reminding us all of that and the other good things for which so many of us are grateful.

Thank you, Dr ozij.

Any helpful (as always) words on the underlying OSA/machine questions from DHC?

[DHC]

Update to the previous data follows.

Upthread the suggestion was made to run the machines on straight CPAP to remove the variables associated with the different vendor's response algorithms. The results of a very brief trial show the following AHI:

Oct 14/15 - AHI 0.2 (ResMed S9 AutoSet w/ResScan 3.11)
Oct 15/16 - AHI 2.98 (DeVilbill IntelliPAP w/SmartLink 2.1)
Oct 16/17 - AHI 3.47 (IntelliPAP)
Oct 17/18 - AHI 0.5 (S9)

Both machines were configured to be straight CPAP supplying 9cmH2O with a 10 minute ramp that starts at 4cmH2O.

With the auto response algorithm not playing a part, the disparity comes down to detection and scoring.

In another thread it was posited that the primary value of the metrics is for trending purposes, in which case, absolute values are less important than values that are consistently produced. I accept that argument, while at the same time, do not believe the question should end there.

Since AHI seems to be an important metric in the field of sleep medicine, and since there are (I believe) published standards for the metric, it does not seem too much of a stretch to expect some reasonable level of universal application of the metric.

I wonder though - what are the possible motivations of a manufacturer to 'fudge' its algorithms to create a patient 'feel-good' factor? Looking around the topics here, it is very common for people to ask someone (especially those who post data values) - well, how do you FEEL? Do you suppose it is possible that one or more of the vendors might be more interested in creating a 'sense' in their patients that they are receiving the best therapy - rather than report the simply and honest data and facts?

My 'sense' is there is little to no difference between the IntelliPAP and the S9 in terms of the 'therapy' they provide. Then again, I am new at this and I have medical issues that may (and probably are) obscuring my ability to 'sense' any differences, hence, my keen interest in the DATA.

When I see such large disparities in those DATA, it arouses my natural curiosity. I remain curious as to the disparity in reported AHI. At the moment, it sure looks to me as though the S9 is under-reporting events. The question now may be - is that intentionally built into the algorithm, or maybe I have a defective machine, or ?

Offered FWIW

[HoseCrusher]

DHC

Keep in mind that in the sleep lab the AHI is scored using a number of parameters. One of those parameters is air flow. The xPAP units only look at air flow and try to extrapolate from that.

In the sleep lab, I believe everything hinges upon arousal's. If your sleep is interrupted by an arousal, the person doing the evaluation looks at the other parameters to see what caused the arousal. In the sleep lab, I am lead to believe that the EEG response is the leading indicator.

Unfortunately, consumer EEG machines are hard to find, and everyone has a little different EEG response, so the interpretation of that data requires advanced training. It is fortunate that there is a loose relationship between air flow and arousal's. The xPAP manufacturers are doing a good job with this single parameter, but I object to their suggestion that a machine scored AHI based upon one parameter is somehow the same as a sleep lab scored AHI based upon several parameters. I prefer to view the machines AHI as an Air flow History Index...

Without getting into a debate about this, it seems that the bottom line is that the fewer air flow interruptions, the better chance for quality sleep. However, perfect air flow doesn't always equal quality sleep. Sleep is complex.

[jnk]

I'll try one more time, then I'll shut up on the subject for now.

. . . In another thread it was posited that the primary value of the metrics is for trending purposes . . .

My manual says: "The AHI values reported by the VPAP Auto should be viewed as trending information only."

Please note the word "only." It does not say "primarily." It says "only."

. . . there are (I believe) published standards for the metric . . .

For PSGs. Not home treatment machines.

. . . it does not seem too much of a stretch to expect some reasonable level of universal application of the metric . . .

That is a stretch for PSGs, let alone home treatment machines, IMO.

. . . I wonder though - what are the possible motivations of a manufacturer to 'fudge' its algorithms . . .

Well, I personally don't particularly like what ResMed did with the S9s either. I would prefer as much trending data as possible, and "O.O" is a difficult place to trend from when numbers can only go up from there, not down. BUT, I still defend their choice in this particular context. I wish the manufacturers agreed on a metric for home machines too, but that doesn't make the disparity an "accuracy" issue, to my way of thinking.

The thing is, people were frightened by the high AHIs the S8s reported, and people used the data to assume and argue that the S8s weren't as effective as other brands. ResMed reacted, because that's what companies do. (Wouldn't you?) They had the class to state openly that they lowered the reported AHI from their machines in order for the reported AHI to be more in line with what a PSG would likely report [that is my paraphrase]. They (1) did not claim they made things more "accurate," and they (2) did not explain exactly how they did it (how they now calculate baseline). I applaud the first choice, and boo the second. But I'm just one guy in the peanut gallery.

Thing is, if their choice keeps a handful of patients using CPAP and their lives are saved, I guess it was the right choice to make. Since one of the biggest problem with PAP therapy is people's feelings about using it, ResMed rightly takes that into consideration with their choices. It might not have been my choice, and I'm frankly still a little miffed about it, but I can live with it, I guess. After all, I am free to keep using my S8 anyway, with it's so-called "over-reporting" that I like so much.

ResMed must feel 'danged if they do and danged if they don't.' If they raise estimates of AHI, for good trending, they get fussed at for it, and if they lower estimates of AHI for peace-of-mind in reaction to the panic caused by the S8s, they get fussed at for THAT! Can't win for losing! (Although market winnings for shareholders seem to be fine and dandy either way.)

Therefore, my recommendation is this:

Buy an S8 if you want to fuss about "over-reporting," and buy the S9 if you prefer to fuss about "under-reporting." But If you want to know what your real AHI is (not just trending-AHI), please get a PSG. That's what they're for, after all.

[DHC]

DHC

Keep in mind that in the sleep lab the AHI is scored using a number of parameters. One of those parameters is air flow. The xPAP units only look at air flow and try to extrapolate from that.

In the sleep lab, I believe everything hinges upon arousal's. If your sleep is interrupted by an arousal, the person doing the evaluation looks at the other parameters to see what caused the arousal. In the sleep lab, I am lead to believe that the EEG response is the leading indicator.

Unfortunately, consumer EEG machines are hard to find, and everyone has a little different EEG response, so the interpretation of that data requires advanced training. It is fortunate that there is a loose relationship between air flow and arousal's. The xPAP manufacturers are doing a good job with this single parameter, but I object to their suggestion that a machine scored AHI based upon one parameter is somehow the same as a sleep lab scored AHI based upon several parameters. I prefer to view the machines AHI as an Air flow History Index...

Without getting into a debate about this, it seems that the bottom line is that the fewer air flow interruptions, the better chance for quality sleep. However, perfect air flow doesn't always equal quality sleep. Sleep is complex.

Understood, and thanks for the response.

The data provided upthread is NOT from a sleep study. It is the result of my having two machines, an IntelliPAP AutoAdjust, and an S9 AutoSet. Since I have both machines, I have been using both of them and gathering data from both. The comparative data on page 1 of this topic and updated today, have nothing to do with the results of my sleep study - though should I have another sleep study in the future, I will certainly LOOK for consistency of data and metrics.

In any case, the data provided in this topic was developed purely from in-home usage of the IntelliPAP and the S9. The two devices produce radically different results (in terms of DATA).

[DHC]

I'll try one more time, then I'll shut up on the subject for now.

. . . In another thread it was posited that the primary value of the metrics is for trending purposes . . .

My manual says: "The AHI values reported by the VPAP Auto should be viewed as trending information only."

Please note the word "only." It does not say "primarily." It says "only."

. . . there are (I believe) published standards for the metric . . .

For PSGs. Not home treatment machines.

. . . it does not seem too much of a stretch to expect some reasonable level of universal application of the metric . . .

That is a stretch for PSGs, let alone home treatment machines, IMO.

. . . I wonder though - what are the possible motivations of a manufacturer to 'fudge' its algorithms . . .

Well, I personally don't particularly like what ResMed did with the S9s either. I would prefer as much trending data as possible, and "O.O" is a difficult place to trend from when numbers can only go up from there, not down. BUT, I still defend their choice in this particular context. I wish the manufacturers agreed on a metric for home machines, but that doesn't make the disparity an "accuracy" issue, to my way of thinking.

The thing is, people were frightened by the high AHIs the S8s reported, and people used the data to assume and argue that the S8s weren't as effective as other brands. ResMed reacted, because that's what companies do. (Wouldn't you?) They had the class to state openly that they lowered the reported AHI from their machines in order for the reported AHI to be more in line with what a PSG would likely report [that is my paraphrase]. They (1) did not claim they made things more accurate, and they (2) did not explain exactly how they did it. I applaud the first choice, and boo the second. But I'm just one guy in the peanut gallery.

Thing is, if their choice keeps a handful of patients using CPAP and their lives are saved, I guess it was the right choice to make. Since one of the biggest problem with PAP therapy is people's feelings about using it, ResMed rightly takes that into consideration with their choices. It might not have been my choice, and I'm frankly still a little miffed about it, but I can live with it, I guess. After all, I am free to keep using my S8 anyway, with it's so-called "over-reporting" that I like so much.

ResMed must feel 'danged if they do and danged if they don't.' If they raise estimates of AHI, for good trending, they get fussed at for it, and if they lower estimates of AHI for peace-of-mind in reaction to the panic caused by the S8s, they get fussed at for THAT! Can't win for losing! (Although market winnings for shareholders seems to be fine and dandy either way.)

Therefore, my recommendation is this:

Buy an S8 if you want to fuss about "over-reporting," and buy the S9 if you prefer to fuss about "under-reporting." But If you want to know what your real AHI is (not just trending-AHI), please get a PSG. That's what they're for, after all.

Jeff,

You have been VERY reasoned (and patient) with your responses. Thank you.

Please do not feel the need to "shut up on the subject" - as it seems there was a time (if my read of your past posts is accurate) when you were looking for similar answers.

I 'get it' that the manufacturers want us - the end users of the devices they manufacture and sell - to consider the metrics less critically. Use of the data for trend purposes is just that - a less critical examination of the metric. I acknowledge that ANY consistent value produced the same way day in and day out will have benefit for trending purposes, and should allow the user to 'tweak' the settings and hopefully improve the therapeutic benefit of the device.

>>For PSGs. Not home treatment machines.<<

Implicit in your response is that the vendors have the freedom to define AHI (as collected and output from their reports) any way they want and whether it comports to any published standard is irrelevant. Please correct me if I mis-stated your point.

While that may be true - it raises a host of other issues. For example, if AHI is malleable, then why is there a standard for its measurement published at all? Moreover, why do doctors and insurances companies use the AHI metric for decisions that directly affect us (the end users - the ones ultimately paying for all this)?

Using an absurd example, if the metric for AHI is not to be followed by the vendors - then why should they bother with the pressure metrics currently defined as cmH2O? They could as easily define it as some derivative of ngogn units (yes, a real measure - sort of), and then further obfuscate the sensibility of USING the data they record and publish. It would still be valid for trending purposes - though have virtually no value to many of the topics posted right here at this forum.

>>Buy an S8 if you want to fuss about "over-reporting," and buy the S9 if you prefer to fuss about "under-reporting." But If you want to know what your real AHI is (not just trending-AHI), please get a PSG. That's what they're for, after all.<<

Well - fussing aside, here is what I want:

* Results from a PSG that are sensible and if followed will improve my quality of life. One of the bits of information I have taken away from this topic is I will insist on understanding the protocols and scoring used for any future sleep study. I will also check to insure they are following the published standards by the AASM.
* A device that follows the same standards as used for the PSG (as much as possible) and produces reports that are at least somewhat 'in the ball park.'
* A manufacturer that produces a set of reports that conform to published standards - which does not prevent them from using a proprietary algorithm for responses or other methods for improved treatment.
* Obviously, I want to 'feel' better - and I want that based on genuine improvement in my health, and not some lies fed to me by a fudged-up algorithm designed to prey on (in fact, influence) my anecdotal responses.

In this topic, early on, there was one person - a ResMed user - who was quick to draw the conclusion that the S9 is providing superior results to the IntelliPAP. Was that person correct? I don't think so - at least, I am not yet convinced. I remain open-minded, but the DATA suggests something different.

[DHC]

Just another thought - and one I plan to try out in the next couple of weeks when I meet with a new sleep doc.

Ask yourself (or try it on your own doc) - what will be the response from a sleep doc to a patient (customer, in my case) walking in to their office for the first time and saying: Doc, I have been monitoring my data on CPAP for some time now and my AHI is consistently less than 1.0 - VERSUS - Doc, I have been monitoring my data on CPAP for some time now and my AHI is consistently between 3.0 and 5.0.

What do you suppose the sleep doc is likely to say in response? Both scenarios describe results less than the 'common' demarcation of 5.0 - so both should be consider "Good" (at least) by most sleep docs.

My hope is (and, as I wrote, I plan to try this very thing), the first thing out of their mouth is to ask WHICH machine I am using for recording data and producing results. I further hope they are knowledgeable enough of the disparities to recognize the differences between the vendor's products. Looking over MANY topics here in this forum, one would be led to believe there are precious few sleep docs with that degree of familiarity.

Leading us BACK to the question of the value of a standard and conformance to that standard.

Again - FWIW

[jnk]

. . . Please do not feel the need to "shut up on the subject" - as it seems there was a time (if my read of your past posts is accurate) when you were looking for similar answers.

I'm still looking for answers on many things. Yes. But I may need to shut up before I get on the nerves of those with more experience and more firsthand knowledge on the subject.

. . . I 'get it' that the manufacturers want us - the end users of the devices they manufacture and sell - to consider the metrics less critically. Use of the data for trend purposes is just that - a less critical examination of the metric. I acknowledge that ANY consistent value produced the same way day in and day out will have benefit for trending purposes, and should allow the user to 'tweak' the settings and hopefully improve the therapeutic benefit of the device.

I'm glad you get the purpose of the data and why it is good enough without being PSG-quality data. That explains why manufacturers are RIGHT in not wanting us, or clinicians, to consider the data more critically--the manufacturers are well aware of the limitations of data based merely on estimates of flow. A lab PSG is a mountaintop astronomy-grade telescope; a home-machine's data is a pair of opera glasses. A lab PSG is an electron microscope, and home-machine data is a drugstore magnifying glass.

Some patients are offended when they talk to the sleep doc about their machine's data and the doc says: "I have no use for home-machine data." I am not offended by that, because from the doc's perspective, the data isn't PSG level and not worthy of being used to make treatment decisions. We can use the data for tweaking, and a big jump in the numbers might catch a doc's attention, but only in the sense that the doc would send the patient for another titration.

What we have to remember is that the purpose of measuring AHI during a diagnostic is mostly just to have something to tell insurance (that the patient has something measurable) and to provide a baseline for the titration AT THE SAME LAB. All that number does, then, is earn the patient the right to try PAP therapy. Plenty of "fudging" of THAT data takes place every day. Then AHI is used to titrate, as well as snoring and other things. BUT not even that lab considers that snapshot to represent the be-all and end-all in diagnosing the patient. Response to treatment gets factored in as well.

So once a patient gets a machine and is titrated, AHI has served its purpose, as far as the medical people are concerned. After that, all that matters is keeping the treatment effective. And that's where home-machine data comes into play. There is no need for it to be very accurate, because the diagnosis and overall titration is over. Now it is a simple matter of answering the question: "Do I increase pressure, lower pressure, or keep it the same?" As long as you have trending data, you have enough info to answer your question. A PSG every night would be overkill, even if you had a sleep lab in your bedroom.

. . . Implicit in your response is that the vendors have the freedom to define AHI (as collected and output from their reports) any way they want and whether it comports to any published standard is irrelevant. Please correct me if I mis-stated your point.

The fact that they do it proves the freedom. The published standards are for the mountaintop telescopes and electron microscopes, not opera glasses and drugstore magnifying glasses, which distort what they measure and have limited measurement capabilities, but are still very valuable at the opera or when organizing your stamp collection.

. . . * A device that follows the same standards as used for the PSG (as much as possible) and produces reports that are at least somewhat 'in the ball park.'

There is no such thing. And there is no ballpark. A home-machine's data is stickball over a manhole cover. Still a lot of fun. (Especially the arguments.) But it isn't major-league ball with an experienced set of umpires.

. . . In this topic, early on, there was one person - a ResMed user - who was quick to draw the conclusion that the S9 is providing superior results to the IntelliPAP. Was that person correct? I don't think so - at least, I am not yet convinced. I remain open-minded, but the DATA suggests something different.

I would tend to trust her judgment in such matters, myself, based on the data put before her, but I wouldn't, of course, let it trump my personal experience in my decision-making for myself on which machine works best for me. And remember, she was discussing indications of the likely efficacy of a particular algorithm with a particular patient (you), and the only real way to know for sure, naturally, is response to treatment in totality, not using home-machine AHI estimates alone. Her pointing out what the data was pointing to was a logical thing to do, since you were asking about the indications from the data. But that doesn't mean she was recommending that the data should be considered more than it is, either.

[jonquiljo]

Ask yourself (or try it on your own doc) - what will be the response from a sleep doc to a patient (customer, in my case) walking in to their office for the first time and saying: Doc, I have been monitoring my data on CPAP for some time now and my AHI is consistently less than 1.0 - VERSUS - Doc, I have been monitoring my data on CPAP for some time now and my AHI is consistently between 3.0 and 5.0.

What do you suppose the sleep doc is likely to say in response? Both scenarios describe results less than the 'common' demarcation of 5.0 - so both should be consider "Good" (at least) by most sleep docs.

Well, they are going to say that you have gotten better control of your apnea if you tell them your AHI is always less than 1.0. I'm going to see my Dr. today and ask her precisely what she thinks (she is definitely smart - so she will be able to give a good response - the jury is still out as to her physician's skills).

I totally agree that "trending" is what you is what's important. BUT, since you are talking reporting that is at least 5 (if not 10) fold different in their numbers - it makes one suspect if any of the numbers can be believed at all. If the speedometer on your car read 30 MPH lower than reality, would you trust it on the freeway to avoid a speed trap? Bad metaphor, but you get the idea.

It's all about this "comfort factor" that I suspect these companies build into their machines - certainly Respironics and Resmed. It looks a lot better to a clinician to see their patient come in with a card full of really low numbers than with a card full of higher ones. The patient feels all comfy when they have the lower numbers too.

If you buy a Porsche - it makes a low and loud grumbling sound as you drive it. If it didn't do that, you'd think you had bought a dud. They can make the car fast and make it quiet too - but why not let the consumer think they've bought a racecar? They really haven't bought anything but a small and fast consumer car - but they can think they have bought the real deal if the manufacturer manipulates it right loud enough to make them happy yet quiet enough to be street legal. They take a lot of time with their marketing and it clearly pays off.

I just question the data - period - when it can vary so much. Trending is good - yes. But get me a little closer to the next brand than a full order of magnitude! Until these machines say something close to each other it really raises a lot of doubts. How many people or Dr's are going to look at the information like DHC just did - no one! Don't underestimate the marketing in this industry. It is getting into such big money that you really have to keep an eye open.

[DHC]

Trying to 'get my head around' this inability or refusal by the manufacturers to report on a consistent basis to a published standard. It is NOT like it would be difficult for them to do so - or is it?

Consider the metric for AHI. Upthread I was told that ResMed calculates it the same way as everybody else. But do they? It seems clear in this case that DeVilbiss and ResMed do not use the same basis. It further seems clear, from the published information made available by DeVilbiss, their reported metrics seem to be inline with the information found at CPAPWIKI pertaining to those metrics.

If it is that the manufacturers prefer to provide a metric that does NOT comport to any published standard and it gives the appearance it is designed to be more of a 'feel good' metric - that sounds a bit like the clothing industry which has been documented to 'size' their garments differently than in past decades so that the customer 'feels good' about slipping into a size (such as it is) that is smaller than they expected. I hope this is not the case, as I surely thought the medical industry worked to a more professional level of ethics/integrity than the garment industry - but now, I am not so sure.

[jonquiljo]

If it is that the manufacturers prefer to provide a metric that does NOT comport to any published standard and it gives the appearance it is designed to be more of a 'feel good' metric - that sounds a bit like the clothing industry which has been documented to 'size' their garments differently than in past decades so that the customer 'feels good' about slipping into a size (such as it is) that is smaller than they expected. I hope this is not the case, as I surely thought the medical industry worked to a more professional level of ethics/integrity than the garment industry - but now, I am not so sure.

This is a much better metaphor for the situation that I could come up with. But I'm afraid with the kind of money involved in all of this you are going to see potentially a lot worse. It's not illegal and I bet most would not consider it unethical - though it makes me question their ethics. Since these are not sold as diagnostic devices, the FDA probably doesn't care at all about what they report to the end-user. Each manufacturer puts in their "disclaimer" and they are free to do whatever they want.

So we are left to look at "trends" - but they are trends of garbage I'm afraid. That's better than nothing, but still a bit short of what we could be getting.

How anyone could look at this and say that these numbers are not disturbing is beyond me. Lots of people when talking about their results on this board say "my numbers look good", etc. Scoring AHI is subjective - yes. But the bottom line is that my machine told me I had an AHI of 0.1 last night and now that number means very little to me - unless I can find a way to see if I can sleep and get a 10.0.

[cwied]

I wonder if anyone has ever done a sleep study using these machines and compared the manually scored AHI to the machine-generated one (assuming that the flow rate measurement on them is accurate enough for PSG)? This would seem to me to be the best way to calibrate the algorithm.

Barring that, presumably if you checked the numbers right after a sleep study (changing as few variables as possible), you could get a rough feel of how to interpret the numbers.