Mattress Ratings >> Mattress Research Sources & Methodology
Mattress Research Sources & Methodology
Page last updated November 2011
The sources for the research data include over 15,000 reviews
/ comments of actual mattress owners collected from diverse sources.
Sources
1. Message Boards / Forums
Over 70% of our research data comes from actual owner experiences collected from nearly 1,000 online message boards or forums.
The boards / forums are usually focused on a specific subject, such as gardening, politics, mountain climbing, astrophysics or just about any other subject you can think of.
But occasionally an off-topic discussion takes place, such as one about mattresses. It is from such discussions that we collect our data.
The boards / forums generally work by someone posting a question and then others reply with answers. In this case, someone typically asks others what mattress they have and whether they recommend it, and then the others respond.
Message boards / forums are generally the best source for getting quality and unbiased mattress reviews from actual owners. An advantage of the boards / forums is that many people who post about their mattress have owned it for a significant amount of time and can comment on the critical issue of mattress durability and lifespan.
In addition, the boards / forums have the important advantage of often existing in near obscurity which means that most do not rank high in the search engines for searches related to mattresses or product reviews. This obscurity tends to ensure unskewed, accurate findings because it largely prevents people from posting who are merely looking for a place to vent about buying a mattress they don't like (or to gush about buying a mattress they do like).
Existing in obscurity also tends to keep out dishonest people who seek to give fictional mattress reviews, such as dishonest people in the mattress industry. In addition, dishonest reviewers are often discouraged by the fact that the boards / forums are usually made up of a tight-knit group of people who keep out suspicious contributors and posts.
2. Retailers' Websites with Unedited / Unselected Customer Reviews
Online retailers such as Overstock, Wal-Mart, Sam's, Costco, Amazon and a few others provide their customers with the ability to rate and review products, including mattresses, purchased from them. We have generally found the reviews on these websites to be credible and unedited / non-selected by the retailer.
The main problem with reviews on these sites is that most of them are written by people who have recently purchased their mattress – as opposed to people who have owned their mattress for a long enough period of time to be able to comment on durability and longevity. When possible, we gather data only from the latter.
3. Product Review Websites
A small amount of our research data may come from established product review websites. These websites allow consumers to post product reviews. Generally, we regard these websites to have only fair credibility mainly because they are so visible and easily found (see "message boards / forums" above for the advantages of being obscure) and because virtually anyone can post a product review. Consequently, in our opinion, these websites' aggregate ratings for a particular mattress can be too high or too low by as much as 40%.
4. Personal Interviews
On occasion, we include owner data collected from personal interviews that we conduct. While the number of these interviews is small, the amount and quality of owner information they provide is often excellent.
Sources Do NOT Include
1. Reviews From Visitors To Our Website
Sleep Like The Dead does not seek or accept mattress reviews from visitors to our website mainly because we cannot verify their credibility. We instead rely on what we regard as more credible sources for getting information from actual owners, namely the online message boards / forums referred to above.
2. Testimonials From Retailer or Manufacturer Websites
Several mattress manufacturer and retail websites feature testimonials from customers. These reviews are often hand-picked by the website operators and consequently are entirely or mostly positive. Also, the truthfulness of such testimonials often cannot be proved. As a result, these testimonials do not have the needed credibility to be included into our research data.
3. Retailer Websites with Selected Customer Reviews
Some retailers provide sections on their websites for customers to review mattresses purchased from them, but they are selective about which reviews are actually published. These review sections often have the appearance of being objective but are likely tilted in favor of the retailer at least somewhat. If we have reason to believe that unfavorable reviews are not being published, we do not include any reviews from the website into our research.
Large Sample Sizes + Diverse, Credible Sources = Accurate Findings
The sample size of at least 15,000 actual owner experiences is large enough in our opinion to provide accurate findings. We at Sleep Like The Dead are not expert statisticians, but we believe that our findings, especially for the owner satisfaction ratings, have a maximum margin of sampling error of ±4 percentage points. We believe this because no matter how much data (information from owner reviews / comments) we add to each mattress type or brand, our findings rarely change beyond the range of a few percentage points.
For a mattress to be included in our main research results (not additional results), we must be able to find at least 150 owner reviews. In addition, in most cases no one source makes up more than 30% of the total number of owner reviews / comments that are included in our data.
Safeguards To Ensure Precise and Accurate Information
To further ensure accurate findings, several measures were taken. Namely, no data was collected from mattress owners who indicated that they have slept on a particular bed for less than one week since this is often not enough time to form a solid opinion one way or the other.
Further, in cases where it was not clear if the reviewer owned the bed, the comments were not included in the data. (Owners were identified when they openly said they owned a particular bed, referred to the bed as “my bed” or identified themselves as an actual owner in some other obvious, non-ambiguous way.)
Information on Owner Satisfaction Rating
The owner satisfaction rating for each mattress type or brand was computed by dividing the number of reviews / comments that showed approval of the mattress by the total number of reviews / comments for the mattress (sample size). A mattress was considered satisfactory by an owner if any or all of the following conditions were met: The mattress was explicitly described as comfortable; the mattress was described as allowing better sleep; the mattress was recommended to others; the owner stated that they like the mattress.