An APAP Shootout (sort of) on Academic Journal

[dsm]

RG

What gets repeatedly missed on these discussions is that there are two perspectives here 1) people who buy and use these machines for their own therapy and 2) medical researchers who attempt to quantify the behaviour of these types of machines so they can either make recommendations to goverments, doctors, and users.

Researchers are doing legitimate work at trying to predict how a bit of medical equipment is going to behave. I know we agree that it is impossible to test every user but surely you do agree that they can strive to come up with some standard recognised patterns of breathing familiar to doctors & researchers. That has been done with the R series of breathing patterns.

Using polls in research is every bit as valid an approach to survey research as using accepted and standardised breathing pattern algorithms in medical equipment research. The point being made was that it is impossible to ask every person their opinion (hence polls) & it is impossible to correlate how an Auto works for every person so standard samples are taken.

But despite this 'circular' debate, you can ignore the reaserch all you like - my doctor didn't and he read it and passes it on to his patients (as I am sure many other doctors like him are doing). He has a string of degrees based on his respiratory specialisation so I have to accept he may know something about respiratory matters. Do I take your 'scientific' research/opinion on research on Autos and pass it back to him and say here is the counter research ? - methinks not

D

[SamCurt]

Since I am a scientist by training (albeit a novice one), I have to say a few things.

Actually, CPAP makers, if they want to model their APAPs as accurately, do have to take the flow patterns of quite some OSA people and summarize that into some patterns (the more patterns the better)-- and then they can use these waveforms to create their iron lungs to test their APAPs. There simply aren't enough people to do this kind of test, which involves, IMO, changing the test APAP for, like, every month, with a PSM test done every week.

The problem is, IMO, CPAP companies didn't disclose what flow patterns they have found such that medical scientists can't study their efficiency. I suppose disclosing that is not as risky as disclosing their algorithms.

[DreamStalker]

RG

What gets repeatedly missed on these discussions is that there are two perspectives here 1) people who buy and use these machines for their own therapy and 2) medical researchers who attempt to quantify the behaviour of these types of machines so they can either make recommendations to goverments, doctors, and users.

Researchers are doing legitimate work at trying to predict how a bit of medical equipment is going to behave. I know we agree that it is impossible to test every user but surely you do agree that they can strive to come up with some standard recognised patterns of breathing familiar to doctors & researchers. That has been done with the R series of breathing patterns.

Using polls in research is every bit as valid an approach to survey research as using accepted and standardised breathing pattern algorithms in medical equipment research. The point being made was that it is impossible to ask every person their opinion (hence polls) & it is impossible to correlate how an Auto works for every person so standard samples are taken.

But despite this 'circular' debate, you can ignore the reaserch all you like - my doctor didn't and he read it and passes it on to his patients (as I am sure many other doctors like him are doing). He has a string of degrees based on his respiratory specialisation so I have to accept he may know something about respiratory matters. Do I take your 'scientific' research/opinion on research on Autos and pass it back to him and say here is the counter research ? - methinks not

D

I understand exactly what you mean Doug. Same thing occurs in my work ... there are those who use our flow models to plan for their future water resources and there those who develop, calibrate, and do sensitivity analysis of those models.

[-SWS]

Respironics: Doesn't seem to respond to the waves

Simple scientific methodology can never be a good methodology when it cannot meet its own designed objectives. This is such a methodology in my honest opinion. I go out of my way to not favor APAP brands, so please don't put me down as coming to Respironics' rescue in this thread or Resmed's or PB's rescue in yet other threads.

I hope everybody reads the methodologically flawed test results I have quoted at the top of this post.

Then please go and read the NR (patient "non-responsive") part of the Respironics algorithm to understand exactly why this testing methodology is absolutely flawed, in that it breaks a crucial patient feedback loop (hint: the "iron lung" machine interfacing with the Respirionics (and other) APAP machines is purely non-responsive. That means any viable algorithm basing at least part of its pressure-response on patient non-responsiveness is doomed, doomed, doomed to fail this miserable testing methodology. Period. End of debate.

And the above is only one of several examples that I can cite that underscore how a broken patient feedback loop skews a comparative APAP testing methodology, when ANY of those algorithms entails a "decision branch" based on a detected state of patient non-responsiveness. But the broken patient feedback loop problem with this methodology truly extends way beyond this simple example. And I could probably carry on ad-nauseum about it. Honestly! .

[dsm]

Respironics: Doesn't seem to respond to the waves

Simple scientific methodology can never be a good methodology when it cannot meet its own designed objectives. This is such a methodology in my honest opinion. I go out of my way to not favor APAP brands, so please don't put me down as coming to Respironics' rescue in this thread or Resmed's or PB's rescue in yet other threads.

I hope everybody reads the methodologically flawed test results I have quoted at the top of this post.

Then please go and read the NR (patient "non-responsive") part of the Respironics algorithm to understand exactly why this testing methodology is absolutely flawed, in that it breaks a crucial patient feedback loop (hint: the "iron lung" machine interfacing with the Respirionics (and other) APAP machines is purely non-responsive. That means any viable algorithm basing at least part of its pressure-response on patient non-responsiveness is doomed, doomed, doomed to fail this miserable testing methodology. Period. End of debate.

And the above is only one of several examples that I can cite that underscore how a broken patient feedback loop skews a comparative APAP testing methodology, when ANY of those algorithms entails a "decision branch" based on a detected state of patient non-responsiveness. But the broken patient feedback loop problem with this methodology truly extends way beyond this simple example. And I could probably carry on ad-nauseum about it. .

[-SWS]

Is the point here that there are no valid tests for Autos ?

No the point of my post above is that this testing methodology is inherently flawed. I'm very much in favor of constructing valid comparative tests. But that is an extremely tough nut to crack, and poorly designed methodology that fails to meet its own objectives simply does not suffice in my honest opinion.

And I do have to say that I am of the opinion that had this report said that the Respironics Auto was the best or had no issues, there would be none of this debate.

I personally take exception to that statement, Doug. It lumps me and my argument in with the rest, as if it was comprised from a pro-Respironics position. My position has always been "pro-patient" in general. And in this thread my position is "anti-bad-methodology" in particular.

This is not a good testing methodology, period. We need one, yes. This is not it.

[rested gal]

*This doesn't mean they don't do a good job for a particular user* (this is RG's issue)

No, that was not my issue or my point. My issue was the lack of usefulness of any test that purports to be comparing autopap responses which does not give the autopap feedback from a dynamically breathing person. Feedback that is absolutely essential for an autopap to do its thing at all.

And I do have to say that I am of the opinion that had this report said that the Respironics Auto was the best or had no issues, there would be none of this debate

[dsm]

Is the point here that there are no valid tests for Autos ?

No the point of my post above is that this testing methodology is inherently flawed. I'm very much in favor of constructing valid comparative tests. But that is an extremely tough nut to crack, and poorly designed methodology that fails to meet its own objectives simply does not suffice in my honest opinion.

And I do have to say that I am of the opinion that had this report said that the Respironics Auto was the best or had no issues, there would be none of this debate.

I personally take exception to that statement, Doug. It lumps me and my argument in with the rest, as if it was comprised from a pro-Respironics position. My position has always been "pro-patient" in general. And in this thread my position is "anti-bad-methodology" in particular.

This is not a good testing methodology, period.

[WillSucceed]

dsm wrote:

If we look at the issues facing researchers in an environment where subjects such as people, have such varied responses and actions it is an outright impossibility to test every single person then produce a scientific report.

When newscasters broadcast opinion polls they take a 'sample' & use selected formulae to come up with a result that usually includes a margin of error. And agreed they don't always get things right but they get them right enough that the world uses polls as better than no input at all and in general the polls have a significant impact on the targets of the polls.

dsm, I think the salient point here is that testing with the iron lung is essentially useless because it does not approximate closely enough the real person. I don't see Rested Gal's comments as "apples and oranges" at all... the manufacturer's would use data from a study like the one's mentioned to "prove" that their machine is better than the competitor's. Based on the data, this MIGHT be true if, and this is a BIG if, the Dr. were prescribing an APAP for an iron lung. But the Dr. is not, he/she is prescribing for a person and the iron lung in no way mimics the respiratory dynamics of a person.

Certainly, as you point out, there needs to be testing and some data evaluating the effectiveness of the different APAP's... the flaw lies not in the desire to generate valuable data, but in the design of the study. These studies offer very little that would be of use in prescribing an appropriate machine for a real patient.

dsm wrote:

if we look at the issues facing researchers in an environment where subjects such as people, have such varied responses and actions it is an outright impossibility to test every single person then produce a scientific report.

Come on dsm, do you REALLY mean to suggest that you think Rested Gal is saying that every single person be tested to produce a valid scientific report? She is just commenting that the testing done in these studies is grossly flawed.

[WillSucceed]

dsm wrote:

And I do have to say that I am of the opinion that had this report said that the Respironics Auto was the best or had no issues, there would be none of this debate

dsm, I'd be just about the first to bash Respironics, but I don't think that Respironics not being the fav. machine in these studies is the issue at all. The studies are flawed, no matter which flow generator is favoured.

[-SWS]

The studies are flawed, no matter which flow generator is favoured.

This test can be thought of as the "flow generator meets flaw generator"!

And yes, I too dislike when everyone jumps on the pro-Respironics/anti-Resmed heap. Put me down as being anti that.

[dsm]

dsm wrote:

If we look at the issues facing researchers in an environment where subjects such as people, have such varied responses and actions it is an outright impossibility to test every single person then produce a scientific report.

When newscasters broadcast opinion polls they take a 'sample' & use selected formulae to come up with a result that usually includes a margin of error. And agreed they don't always get things right but they get them right enough that the world uses polls as better than no input at all and in general the polls have a significant impact on the targets of the polls.

dsm, I think the salient point here is that testing with the iron lung is essentially useless because it does not approximate closely enough the real person. I don't see Rested Gal's comments as "apples and oranges" at all... the manufacturer's would use data from a study like the one's mentioned to "prove" that their machine is better than the competitor's. Based on the data, this MIGHT be true if, and this is a BIG if, the Dr. were prescribing an APAP for an iron lung. But the Dr. is not, he/she is prescribing for a person and the iron lung in no way mimics the respiratory dynamics of a person.

Certainly, as you point out, there needs to be testing and some data evaluating the effectiveness of the different APAP's... the flaw lies not in the desire to generate valuable data, but in the design of the study. These studies offer very little that would be of use in prescribing an appropriate machine for a real patient.

dsm wrote:

if we look at the issues facing researchers in an environment where subjects such as people, have such varied responses and actions it is an outright impossibility to test every single person then produce a scientific report.

Come on dsm, do you REALLY mean to suggest that you think Rested Gal is saying that every single person be tested to produce a valid scientific report? She is just commenting that the testing done in these studies is grossly flawed.

[-SWS]

I said that testing all human responses is impossible - I posited in one of my responses the question of what was a normal number of people to test - perhaps you missed that.

Think Statistics 101.

What tests do you believe researchers might devise. ?

Having a little background in R&D testing methodology, I would personally write a several-hundred page test plan, and spend several months doing it.

Simple is best only when simple can suffice, and I honestly don't think simple suffices here.

[dsm]

I said that testing all human responses is impossible - I posited in one of my responses the question of what was a normal number of people to test - perhaps you missed that.

Think Statistics 101.

What tests do you believe researchers might devise. ?

Having a little background in R&D testing methodology, I would personally write a several-hundred page test plan, and spend several months doing it.

Simple is best only when simple can suffice, and I honestly don't think simple suffices here.

Does this bring us back to the position that there are no adequate tests for researchers to test Autos and come up with recommendations on performance & deployments ?

I come back to the point made earlier that among the world's respiratory specialists, this report was published worldwide by a respected US medical magazine. Now either the respiratory profession lives in a world of flawed research into Autos and acts on it or we are being to narrow in our view of this report and are arguing apples & oranges in regard to research vs individual use of Autos.

Cheers

D

[-SWS]

No, Doug. It brings us to the point that no adequate methodology for overall APAP comparison has been devised yet---and that measuring a faint, cosmic RF signal with a ten-cent wooden ruler is not good methodology simply because it is the only methodology we have been able to devise. Even if that ten-cent wooden ruler shows up in a scientific magazine.

How much bad methodology has shown up in scientific publications throughout the years? Plenty.

I come back to the point made earlier that among the world's respiratory specialists, this report was published worldwide by a respected US medical magazine. Now either the respiratory profession lives in a world of flawed research into Autos and acts on it or we are being to narrow in our view of this report and are arguing apples & oranges in regard to research vs individual use of Autos.

We certainly can't apply that validation criteria to independent scientific studies that contradict each other. The statement becomes logically broken as a universalism or generalized criteria in the case of highly-respected contradictory studies. If the argument breaks as a universalism, it logically cannot be adequate basis for validating any one scientific study---including this study. And the argument certainly can't be used to validate all those other historically proven bad scientific studies as well.