CPAPtalk.com

Zeo has a very hard time telling the difference between one of the sleep stages and being awake

Rem stage and wake are both higher brain wave activity states. I sometimes see the same wake to Rem with nothing in between. I can usually tell from the flow rate patterns whether I'm dreaming or awake.

To OP, I concur with what others have suggested. Try it on someone else. Lend it to friend in a different environment. If it still misbehaves it is a sure bet the one or both parts are defective. Try it on someone else in your home environment. If it works for them it is probably not RF interference and more probably related to interpreting you.

BasementDwellingGeek wrote:

Zeo has a very hard time telling the difference between one of the sleep stages and being awake
Rem stage and wake are both higher brain wave activity states. I sometimes see the same wake to Rem with nothing in between. I can usually tell from the flow rate patterns whether I'm dreaming or awake.

Two comments, both slightly in jest. Your nights record even worse than mine, and I have noticed the difficulty in determing sleep state.

I had to get new headbands within a month, due to sweat degrading the readings. I also noted that if it wasn't centered right-left, it wouldn't record accurately at all.

Finally, is there any chance a pet is sneaking in and wearing it during the night?! Looks like my dog's usual nightime prowling around
Seems you need to get Zeo to check out the unit.

jwerley wrote:Has anyone had problems with their Zeo's not being accurate....

See the Zeo company's own "Behind the Headband" Youtube series (http://bit.ly/OMoGVa). According to their own stats the Zeo is only 64% reliable.

The details... they claim that polysomnographs (PSG) are only 85% reliable. They further claim that the Zeo is 75% as reliable as a PSG. Multiply the two together and you get 64% reliability for the Zeo.

Merely flipping a coin is 50% reliable and that's where we're heading here.

tanaats wrote:According to their own stats the Zeo is only 64% reliable.

Accuracy I think you mean. This is in regards to the classification of sleep states. This is one reason I use a hacked bedside (their hack, not mine) and a serial cable and record my brainwaves, because they are what they are and tell me more info than merely a classification algorithm, the classifications are man-made anyway, for convenience, not fact. What I get in the end is this, the top being brainwaves, the bottom being the classification and my eye can judge where the classifications are "iffy".

"Accuracy", yes, thank you. It doesn't completely break 64% of the time.

In the same video they say that an EEG channel on the forehead can't always be used to distinguish between REM and wake. This seems to me like a serious problem with their core technology that can't be overcome by analytical methods.

tanaats wrote:In the same video they say that an EEG channel on the forehead can't always be used to distinguish between REM and wake. This seems to me like a serious problem with their core technology that can't be overcome by analytical methods.

Discriminating between REM and Wake is well known to be very difficult, particularly for a device that costs less than $150. In the chart above I woke up at 5:45 AM, this I know for a fact, I had a major nose itch and had to pull away my mask for a few seconds and my flow spiked so I can see when it happened and I can see the Zeo classified it as a bar of "REM". The bars of Wake and REM at the end could well be close, as I was coming out of sleep naturally, in a process of waking / dreaming / waking.

I'm quite satisfied with the information I get. It is probably the most useful for measuring my sleep quality for the money.

You are a quite highly sophisticated user of the Zeo data. For someone who just uses a Zeo out of the box as instructed I'm not sure at all of its usefulness. I've read a few comments on various forums by people agonizing over their Zeo data. They may be agonizing over nothing. 64% is just too close to a coin flip.

Tanaats, I believe Zeo claims 75% overall accuracy, not 75% of PSG accuracy. See http://blog.myzeo.com/behind-the-headba ... cy-of-zeo/

That being said, I think many of us agree the value of Zeo is not in its ability to accurately determine the events of any one night, but rather the trend in events over time as a therapy is applied.

Jay

This debate has been rehashed many, many times here. There's a poll here where Zeo users state to what degree they like it. It's a clear affirmative.

I hope the discussion isn't closed.

"Anecdoctal evidence" such as a poll can be unreliable. An accuracy rate of 63% would indeed result in a clear affirmative, for example. Then there is "sample bias", i.e. the people taking the trouble to vote here on a CPAP forum may not be representative of the entire population of Zeo users. And for anyone having already bought a Zeo a "confirmation bias" factor can also kick in (we are all subject to it). There are lots of other reasons why medical researchers (disclosure: I am not in the medical field) don't rely on anecdotal evidence.

For anyone to start taking Zeo data as gospel only because a clear majority of people who took the trouble to vote here like the Zeo isn't a good idea. No matter how people vote here in favor I think that anyone using a Zeo should be aware that the accuracy is down towards a coin flip, and this is by the Zeo company's own statement.

tanaats wrote:I hope the discussion isn't closed.

Never is around here.

"Anecdoctal evidence" such as a poll can be unreliable.

Correct. And this is a subset of the Zeo community, ones with sleep breathing disorders.

For anyone to start taking Zeo data as gospel

Formation of a "strawman" argument. Very few here take the Zeo as gospel. Anyone who does is an exception.

the accuracy is down towards a coin flip, and this is by the Zeo company's own statement.

The company's statement is 75% accuracy. That's not a coin flip.

You are a quite highly sophisticated user of the Zeo data.

Maybe, but the most "sophistication" I had to do was make a 3 wire serial cable. Zeo does not sell one (they should). The instructions are somewhat clear. Most people may be challenged with that, I was, because I'm not a hardware guy. The firmware update to the Zeo was trivial easy and there is free software for viewing and storing your brainwaves ("ZeoScope"). The chart I show is from our software, only because I put it with my other data.

If you don't think it's useful, don't get one. I think it's useful. I got one and find it to be very useful. Am I biased? Sure, I like things that are useful.

tanaats wrote:

jwerley wrote:Has anyone had problems with their Zeo's not being accurate....

The details... they claim that polysomnographs (PSG) are only 85% reliable. They further claim that the Zeo is 75% as reliable as a PSG. Multiply the two together and you get 64% reliability for the Zeo.

No, that's not what they're saying in this video - https://www.youtube.com/watch?v=jMmCHXSKI9U - nor is that what probability theory indicates.

The 85% figure comes from the fact that two well trained individuals scoring a PSG will score it differently. This is human error, not statistical error, and it's incorrect to use the standard formula for computing two probabilities with it. Any standard statistics textbook will explain this to you if you're confused about it. That is a very far cry from the random walk you are suggesting.

About 75% of the time, a Zeo's scoring algorithm will produce results within a standard deviation of the accuracy of a PSG being scored by humans. The other 25%, it won't. In the case of the video, the guy mentions his night in a sleep lab with a Zeo was about 99% correlated to a PSG of the same night. Other times, it's been 1%. If you look at the actual studies they've put out, this is what is they've written as well. It's also worth noting that they claim it is probably about as accurate as most at-home sleep study devices.

On its own, the Zeo isn't a good diagnostic tool. It has major issues with tracking awakenings, since it doesn't record them on the algorithm unless they're over 2 minutes. As best as I can tell, it doesn't know how to even record a CAP pattern in the raw data, which can leave out multiple short term awakenings. There can be measurement error when you don't keep the headband clean. It doesn't record a lot of other data that are very helpful for determining sleep quality, like position and such. And in sum, even if it gets perfect data it sometimes misinterprets it.

These are certainly issues with both the algorithm and the measuring device, and someone like MaxDarkside probably solves many of them by skipping the algorithm to play with the raw data. I haven't done that since I prefer the utility of being able to sneak a very quick look at my phone when I first get up to determine if I've had a good night or a bad night when my body isn't ready to tell me the same thing yet. Sometimes the Zeo will be wrong, but most of the time it's right.

However, it has great utility over time as part of an overall trend of how you're sleeping, since it reveals a lot of material that until now has been the province of a sleep lab. In combination with other monitoring tools, it is extremely helpful for diagnosing stuff that doesn't show up in AHI or pulse ox data but does show up over a long term trend. This is why we were suggesting the original poster here return the headband, since her Zeo data don't correlate with what other sources of data are suggesting; that leads most of us to think she's just got a bad headband.

If you don't like the product, don't buy it, but many of us (including those of us with medical and statistical training, like myself) find it one hell of a toy especially at this price point.

MaxDarkside wrote:This debate has been rehashed many, many times here. There's a poll here where Zeo users state to what degree they like it. It's a clear affirmative.

FWIW: The sleep tech I had for my sleep study last night was chatting with me this morning about various things (besides my crummy sleep study). I don't have a Zeo, but he mentioned that at his medical center the docs were curious a while back and did some experiments with their patients to see if the Zeo correlates well with the data from the patient's polysomnograph study. He said the correlation was pretty impressive.

I just posted this. Are you having the same problem???

Starlette

viewtopic/t80863/Zeo--Where-did-My-Data-Go.html

old64mb wrote:someone like MaxDarkside probably solves many of them by skipping the algorithm to play with the raw data.

Right, because the classification algorithm (both in the Zeo and in a PSG) is a discretization (in the sense of statistics and machine learning) of a continuous natural process. Sleep stages exist for convenience to quantify the continuous values into a simple set of metrics. Substantially more information is in the continuous data. For example, our brains generally do not snap from one sleep stage to the next like we see in a hypnogram, but transition in an ebb and flow. I can see the transitions and "fuzzy states" and make my own judgment along with what the Zeo classification says.

The other angle to this is regarding accuracy vs. repeatability. I've said it here before... Something can be wrong but still be very useful if it is reasonably repeatable. When people here say they watch the "trends" they get from Zeo, this is what they are expressing, perhaps without realizing. An example in this case is ZQ. There is no standard definition for a sleep score, but Zeo creates one, so by definition it is neither accurate or inaccurate but accuracy does not matter... regardless if you agree or not on how it's computed, its trend is very useful because it has meaning and is generally repeatable.