Is ZEO accurate?

The Zeo dudes give a historical overview of sleep staging/scoring, explaining the N3-N4 combining by the AASM:

http://www.myzeo.com/sleep/knowledge-ce ... ge-4-sleep

Some further thoughts on not worrying too much about deep sleep, also from the Zeo dudes, according to their view:

http://www.myzeo.com/sleep/knowledge-ce ... deep-sleep

(Shhhh! They said don't talk about it.)

Another interesting view on sleep staging/scoring, as the Zeo dudes linked to it, if you want to go deep--as far as deep info, not deep sleep. (Am I still allowed to talk about anything being "deep" now? Or is that against the rules? I don't know. Shhhh!):

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2335403/

WoW, This discussion is getting rather deep, but I will take a look at the links when time permits.

MaxDarkside, you should like this.

jnk wrote: ...Another interesting view on sleep staging/scoring, as the Zeo dudes linked to it, if you want to go deep--as far as deep info, not deep sleep. (Am I still allowed to talk about anything being "deep" now? Or is that against the rules? I don't know. Shhhh!):

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2335403/

Jay Aitchsee wrote:MaxDarkside, you should like this.

Yup. There's a lot that can be done to better characterize sleep. I could ramble on for hours...

Is ZEO accurate?



even by their own admittance, no it's not very accurate! Most quote the 75% number above, but that is inaccurate. It states, "75% as good as PSG which is only 85% accurate. Do the math, 75% of 85 is less than 64%. To, me you might as well make a guess yourself and flip a coin to see if your right. You have a 50% chance of being right. That means your method is more than 78% as accurate as their ZEO method. Doesn't my 78% number look impressive!

When it comes to marketing, they can word numbers to sound really good. So send me your info, I'll flip a coin, and make my best guess, and I'll do this for a mere $3.99. Save yourself around $90. That is all the ZEO is, it's the next marketing gimmick! Like the next super high speed blender, or super suction vacuum cleaner! But, it's working, look at all the people that have been bought this unit, right here on this site. Because everyone wants the latest and greatest gadgets! Then some want to believe that the info they receive off of this unit, is more accurate than a PSG study! Watch the video, like they state, there are to many other variables, not calculated into the unit, to be accurately reliable! I think this unit may be trying to sell a lot of false hope. The unit, with a lot of work, may become a very informative device in the future, but it's not there yet!

If you feel it is giving you reliable and informative information, good for you, I'm glad your happy with it. But, please don't try to hype this as being better and more accurate than a PSG!

Everyone is entitled to their opinion. In my opinion they are not playing any marketing games. When I first got my Zeo, admittedly I was hesitant on the reliability of the figures. I've become more trusting of what it's showing me over time.

Disclaimer of course: YMMV

I believe Zeo to be respectably good at doing what it does, although not as good at doing what it isn't even trying to do, even though some may wish it did that too.

I just reread my preceding sentence and no, I'm not sure what it means either; but I meant it a few seconds ago when I wrote it, so I'll let it stand and maybe I'll figure out what I meant later.

Compared to perfect, everything is crap. The Zeo does remarkably well and is useful for a broad cross-section of people for only having three sensors on your forehead. Kudos to them.

MaxDarkside wrote:Compared to perfect, everything is crap.

I would like a T-shirt that says that, please.

lazer wrote:Everyone is entitled to their opinion. In my opinion they are not playing any marketing games.

Like you said, "Everyone is entitled to their opinion". Anytime you state numbers like are shown in the graph in the above post, 75% of 85% is marketing games. Why? Because if they said outright that their unit has an accuracy rating of <64%, not many people would buy it. Those numbers (75% of 85%) are their own numbers, not ones I pulled out of the air. Then, you have to consider, when a company is in business to sell a product, they will sometimes skew some numbers in their favor. The unit runs off a human designed and interpreted algorithm, so it can only be as accurate as the info fed into the design. If the information fed into the algorithm is flawed, then so are the results.

In #4 above, We don't measure movement, breathing, etc. and can't pick up sleep disorders. We all know that the brain waves of someone with sleep disorders, is a lot different than a normal person. In #5 above, they use a single channel sensor, where I believe PSG uses at least 3 channels to measure brain waves and eye movements.

Basically, what they are saying is we can tell when the brain has went into a deeper sleep because of the Delta wave activity, and that is about the gist of what the unit does. The rest of the time is just a guess at best!

If you listen to the video, they have run 3 internal studies, internal, meaning they can come up with results that fit their needs. In the above study, they studied 10 people, of which 4 were females and all had no sleep complaints. Again, I have to question the validity of their studies. Ten people is not much of a sampling, and again, did they only test 10 people, or is that the 10 best results they pulled out of the test? And why not test it on people with sleep problems? He even states in the video, that when he used it, his test come back at 99% accurate, but that other peoples test had come back as low as 25 to 30%. That most people would fall within that spectrum. I'd hate to think, that I was relying on this device to give me an accurate report, but that I was one of the fortunate ones that was at the 25 to 30% range, and I have no way of knowing that. If the device is as good as they state, 90% accuracy on determining sleep levels, how can it only be 25 to 30% accurate on some people? If the algorithms are programmed correctly and accurate, there shouldn't be people down in the 25 to 30% range.

Artificial intelligence (the secret ingredient) a computer program written by a human that uses formulas that is input into the software. If the formulas are inaccurate, so are the results! It needs to be tested more thoroughly, to test its accuracy!
I'm not saying it's a worthless device, I'm saying not to rely on the results as gospel! Several people are saying the device is showing that they are getting the restful sleep they are supposed to get, but they still feel tired. It could be that they are not receiving accurate information! Others say the information is very accurate, how do you know this? Because you wake up feeling good, even people with untreated OSA wake some mornings feeling good! I just want people to go buy this device with their eyes open, and not think, "It will be accurate for me and I can believe everything it tells me". Personally, If you have the extra money and the cost is no big deal, go for it! But, if your on a tight budget, I would recommend you save your money! Most trials out there pay you something for your time and provide you with the products needed for the trial. But, ZEO has people buying their product and using them as guinea pigs to get their feedback about their product! Kind of reminds me of those black 8 balls people bought years ago, you ask it a question and turn it over for your answer! Sometimes it's right, sometimes its wrong!

woodworkerjunkie wrote: So send me your info, I'll flip a coin, and make my best guess, and I'll do this for a mere $3.99.

Now if you set yourself up as a service, and offer this for $3.99 a month, you just might have found yourself a nice part time job!

If I could get all wired up, hire a personal sleep tech, and have a PSG every night, I would have some awesome data. The Zeo clearly isn't as good as that. Does it provide "useful" information? That is in the eye of the beholder. I own one, so clearly, I have some bias, but I find the results interesting.

Mainly, I think it is useful for trending. If I wore it every night for 3 months, I would start to see trending info. That info could be useful.

For the past 556 nights (minus 5 or 6) I've used a simple motion-based sleep clock on my iPhone. It is certainly not as detailed as even the Zeo, not to mention a full sleep lab, but with the data I've collected over 556 nights, it provides useful feedback.

http://www.sleepcycle.com

So, yes, people need to know what they are getting, and what they are not getting if they purchase a Zeo.

John

JohnO wrote:
I own one, so clearly, I have some bias, but I find the results interesting.

Mainly, I think it is useful for trending. If I wore it every night for 3 months, I would start to see trending info. That info could be useful.

For the past 556 nights (minus 5 or 6) I've used a simple motion-based sleep clock on my iPhone. It is certainly not as detailed as even the Zeo, not to mention a full sleep lab, but with the data I've collected over 556 nights, it provides useful feedback.

http://www.sleepcycle.com

So, yes, people need to know what they are getting, and what they are not getting if they purchase a Zeo.

John

I agree with you, it can provide useful info! But, to many people are buying this unit as the absolute answer to their questions, and it is not going to be factual information they can rely on. Some are even claiming that it's more accurate than their PSG sleep study, and in a very select few cases that may be true, but not many. It can give them some clues and possibly help them change their sleep habits. There just seems to be a lot of people posting, making it sound more accurate, than even the owners of the device claim it to be! I hate for people to spend money on an item, and find it doesn't give them the real information they were hoping for or worse yet info that is inaccurate!
Give the company a few years and they may have a device that will be able to give accurate results and be able to network several devices together for an accurate in home sleep study. To me, they are still just a little to far off for me to want to mess with it, and I think a company should be straight forward with their information. Don't hide behind smoke and mirrors! Some peoples lives may depend on the info they are providing, if people change things based on faulty information!

The link you provided looks interesting, I wonder if they have a app like that for the droid?

woodworkerjunkie wrote: . . . when a company is in business to sell a product, they will sometimes skew some numbers in their favor. . . .

Hey, I love a good conspiracy theory same as the next guy. But your stating that some companies will do it is not proof Zeo has. In my opinion. I agree we can never swallow what a company says without being skeptical. But compared to other companies, I think Zeo has shown more than an 85% instinct to be forthcoming with their product's limitations.

woodworkerjunkie wrote: . . . runs off a human designed and interpreted algorithm, . . .

I have found that the Martian ones, as well as the ones based on other alien technology, donn't seem as effective at measuring human biological functions, unless specifically designed to do so when used after abducting humans for biological testing in the mothership. But hey, that's just my take. Other than that, all machines are human designed and all results are interpreted by humans. Or should be. I curse the robots!

woodworkerjunkie wrote:. . . In #4 above, We don't measure movement, breathing, etc. and can't pick up sleep disorders. . . .

In my opinion, the words you chose to quote are proof of exceptional honesty of the Zeo dudes and particular scientific straightforwardness that smacks of bad marketing if anything, not dishonestly good marketing. Those are things I would assume that they would not say out loud if the purpose was to pull the wool blankets over anyone's REM-darting eyes.

woodworkerjunkie wrote: . . . We all know that the brain waves of someone with sleep disorders, is a lot different than a normal person. . . .

Apparently. Moreso for some than others, perhaps? Ahem. But since the Zeo dudes say it isn't designed to detect, treat, or be used in conjunction with treatment of sleep disorders, I don't see how your statement indicts them in the slightest. It only indicts the expectations of those who have expectations that were not created by the Zeo dudes about their product. As I see it.

woodworkerjunkie wrote: . . . they use a single channel sensor, . . .

Simple and cost-effective for what it is designed to do.

woodworkerjunkie wrote: . . . The rest of the time is just a guess at best . . .

Is that your best guess? I think charting data is educated guesswork in the best of instances for most any endeavor that isn't simple black and white on or off.

woodworkerjunkie wrote: . . . If you listen to the video, they have run 3 internal studies, internal, . . .

Who else would study the product? They actively welcome outside independent verification. That's the best any company can do.

woodworkerjunkie wrote: . . . meaning they can come up with results that fit their needs. . . .

Can, yes. Or they could be poisoning our minds by inserting brain waves while we sleep. That's why I wear an aluminum-foil hat. But hey, that's just me.

woodworkerjunkie wrote: . . . how can it only be 25 to 30% accurate on some people? . . .

They are studying humans. Humans differ. That is the nature of scientific study of humans. The reports are made for what works for most people. Penicillin is highly effective. But not on every disease and not in every human. It can kill some people. But it is the thing that it does for most in some specific circumstances that makes any product what it is. (Maybe a bad analogy, since Zeo is not a medical product, just one for "sleep coaching." Anyway . . .) Some humans' brain waves are easier to report than others. Zeo says so honestly. Kudos to them. They could have hidden that info had they wanted to.

woodworkerjunkie wrote: . . . If the formulas are inaccurate, so are the results! . . .

Which formulas would those be? The formula used by the product, or the formula being used to review its ability to do what it was never designed to do? Just asking.

Woodworker, this is what they actually say:

http://blog.myzeo.com/behind-the-headba ... cy-of-zeo/

The Gold Standard
The current gold standard for sleep has an accuracy of 85% out of a possible 100%. Given all the bodily functions that need to be monitored in order to perfectly determine sleep/wake and sleep phases–not to mention the fact that everyone is different– it’s highly unlikely that we’re going to hit that 100% with a one-size-fits-all method or procedure.
In terms of “grading on a curve” that 85% now becomes an A+. It’s the new standard that other methods need to be compared against.
The technology behind Zeo means that based on the fictional 100%, we’re at 75% accuracy. But when you compare it to the actual ceiling of 85% , we’re much better. When you work it out, Zeo is 88% as good as the current gold standard. That’s a B+ on our grade curve.
That said, since the metric that we’re fine-tuning our system to is only 85% as good as a hypothetical 100%, there are going to be outliers. People are going to get different results due to small biological differences and due to the natural limitations of that 85%. But that does not affect the overall accuracy. When tested repeatedly, that B+ accuracy will stand as a general rule.
Know Thyself
What this all means for you, the Zeo user, is that your readings and results are your own. They are a reflection of how your body works and interacts with Zeo. Sometimes, your readings could mean that you’re an outlier, but again, those are just based on averages as well.
Instead of getting hung up on the one offs, take comfort in knowing that there are a number of ways in which your brain signals and activity are similar to others. Statistically, that’s what that B+ accuracy means. It means that 8.8 out of 10 users’ brains will be similar enough for Zeo to know what phase of sleep you’re in.
What you then do with those numbers–and more importantly, what they mean to you–is for you to decide. Don’t worry, we’ll continue to give you real, science-based information to help you out, but your still going to make the final decisions.
Some will look at 30 minutes in Deep sleep and think it’s terrible; others think it’s great.
One camp thinks a more holistic and balances approach to life is the way to go, but others are hell bent on being as efficient and precise as possible.
Some will even think that based on the B+, Zeo sucks. They want only an A or better. That’s fine, too–but a sleep lab night after night is awfully expensive, not to mention uncomfortable.

Jay

Woodworker,

I think you are asking all the right questions and pointing out all the major concerns, and I think your posts are moving the discussion along nicely.

And I'm not just saying that because I had many of the exact same concerns myself, as expressed in this old thread: viewtopic.php?f=1&t=42560&st=0&sk=t&sd=a#p375048

As Derek from Zeo said back then in that thread:

Derek@Zeo on Fri Jul 24, 2009 3:39 pm wrote:At occasional points in time, Zeo can have some difficulty in determining if you are in Wake, Light sleep or REM sleep – but the aggregate data (your total Z, time in REM, time in Deep over the course of the entire night) compares very well with the results of a PSG (the gold standard for assessing sleep).

I have no argument with his wording, myself, but I can understand why some might.