A Blind Forced-Choice Study of Astrological Accuracy

Most astrology companies would never commission a study of their own accuracy. The commercial incentive runs in the opposite direction: ambiguity is profitable, and precision is a liability if precision can be measured and found wanting. Beaufort Intelligence takes the opposite position. If the analysis is accurate, a well-designed study will demonstrate it. If it is not, continuing to sell it without knowing is indefensible.

The existing literature on astrological validity is sparse and methodologically contested. Carlson's 1985 double-blind test, published in Nature, remains the most cited study and is widely treated as definitive evidence against astrology.^[1] But Ertel's 2009 reanalysis demonstrated that Carlson's own data, when analysed according to the effect-size criteria Carlson himself specified in advance, showed a statistically significant result in favour of astrological accuracy, a finding Carlson's original paper did not report.^[2] The study that was supposed to close the question left it open. The accompanying literature review documents this pattern in detail.

The problem is not that astrology has never been tested. The problem is that it has rarely been tested well, and when it has, the results have been interpreted through the lens of the researcher's prior commitments rather than the data. Ioannidis demonstrated in 2005 that most published research findings are false, a consequence of small sample sizes, small effect sizes, financial and other interests, and flexibility in study design and analysis.^[3] The Open Science Collaboration's 2015 attempt to replicate 100 published psychology studies found that only 36 percent produced statistically significant results on replication.^[4] If mainstream psychology cannot replicate its own findings at above two-thirds failure rate, the evidential standard applied to astrology must be at least as rigorous, and the scepticism about existing negative findings at least as sharp.

This study is designed to meet that standard. It is specified publicly, will be pre-registered before any data is collected, and the results will be published in full. The methodology is documented here so that it can be evaluated, challenged, and improved by anyone with the expertise to do so.

The primary research question is precise and falsifiable: can participants identify astrologically derived natal readings as more personally accurate than plausible matched-but-randomised readings at above-chance rates?

If participants rate the five readings derived from their actual natal chart significantly higher than the five derived from a randomised chart, that constitutes evidence of discriminative accuracy. If they do not, that constitutes evidence that the readings are not distinguishable from well-constructed alternatives, regardless of how personally meaningful any individual reading may feel.

The secondary research questions address the variables that any honest analysis of the data must account for. Does prior belief in or experience of astrology predict accuracy ratings independently of reading source? This tests for confirmation bias. Does the specificity of a reading, how tightly it is tied to a particular natal configuration rather than a general psychological statement, correlate with accuracy ratings? This tests whether astrological specificity adds discriminative value beyond the Barnum effect, the tendency for people to accept vague personality descriptions as uniquely applicable to themselves, first documented by Forer in 1949.^[5] Does the motivation for participation affect the distribution of ratings? Participants incentivised by a cashback offer may engage differently from those motivated by genuine interest in the research.

The study uses a within-subjects blind forced-choice paradigm. Every participant receives both real and randomised readings, eliminating between-subjects variance and providing maximum statistical power per participant.^[6]

The design controls for several known threats to validity. The within-subjects structure means each participant serves as their own control, eliminating individual differences in rating tendency. The matched-era randomisation ensures that generational planet positions (Saturn, Jupiter, Uranus, Neptune, Pluto) are approximately consistent between real and randomised charts, preventing participants from distinguishing readings based on generational markers alone. The randomised presentation order controls for primacy and recency effects. The blind labelling prevents demand characteristics: participants cannot preferentially rate readings they believe to be real.

The Barnum effect is addressed directly by the design of the randomised readings. Rather than using generic personality statements as controls, the randomised readings are generated from an actual natal chart using the same analytical system. They are therefore astrologically specific, just not specific to the participant. If participants rate the real readings higher than the randomised readings despite both being astrologically detailed, the discriminative signal cannot be attributed to the Barnum effect, because the control condition is equally specific.

The primary analysis will use a paired-samples test comparing mean accuracy ratings for real readings against mean accuracy ratings for randomised readings within each participant. Cohen's guidelines for power analysis inform the sample size requirement: detecting a medium effect size (d = 0.5) at alpha = 0.05 with 80 percent power requires approximately 34 participants in a within-subjects design.^[7] The target sample size is set at a minimum of 100 participants to allow for subgroup analysis and to provide sufficient power for detecting smaller effects.

Secondary analyses will examine the relationship between prior belief, reading specificity, and participation motivation on accuracy ratings using mixed-effects models. The belief variable is of particular importance: if participants who believe in astrology rate all readings higher than participants who do not, regardless of source, that constitutes evidence of confirmation bias rather than discriminative accuracy. Conversely, if the real-versus-randomised difference is consistent across belief levels, the effect is independent of prior commitment.

Exploratory analyses will examine time-on-page data to identify rushed responses, which may indicate disengagement, and whether response patterns differ by report type (shadow work, personality, love, and so on). These analyses will be clearly labelled as exploratory and will not be used to make primary claims.

All statistical decisions, including the primary test, alpha level, effect size threshold, and sample size target, will be specified in the pre-registration document before any data is collected. Post-hoc changes to the analysis plan will be documented transparently as deviations from the registered protocol.

The study will be conducted in accordance with the British Psychological Society's Ethics Guidelines for Internet-mediated Research, which provide the authoritative framework for online psychological research conducted from the United Kingdom.^[8] These guidelines require informed consent, the right to withdraw, transparent data storage, and appropriate debriefing, all of which are built into the study protocol described above.

The informed consent process will explain the purpose of the research, the nature of the task, what data is collected, how it is stored, and the participant's right to withdraw at any time without consequence. Participants will be told that they are participating in a research exercise, not receiving additional analysis. They will not be told in advance which readings are real and which are randomised. This is standard blind methodology and does not constitute deception under BPS guidelines, as the participant is fully informed about the purpose and procedure, and the blinding is essential to the validity of the design.

The research ethics framework also draws on the principles established by the Declaration of Helsinki, which requires that research involving human participants must be conducted only by individuals with appropriate scientific training, that the wellbeing of participants must take precedence over the interests of science and society, and that every participant must be adequately informed of the aims, methods, anticipated benefits, and potential risks of the study.^[9]

Data storage will comply with the UK General Data Protection Regulation and the Data Protection Act 2018. All data will be stored in encrypted databases with access restricted to authorised personnel. Published datasets will be fully anonymised: no personally identifiable information, no birth data, and no information from which a participant could be identified. The Information Commissioner's Office guidance on data protection in research contexts governs the retention and anonymisation standards.^[10]

Beaufort Intelligence is a commercial operator with a financial interest in positive findings. This is acknowledged explicitly. The pre-registration, blind methodology, and commitment to publishing regardless of outcome are designed to mitigate the risk that commercial interest influences the interpretation of results. But the conflict of interest cannot be eliminated, only managed. It will be disclosed in all published findings, and the study is designed so that any qualified researcher can replicate it independently using the published protocol.

The most significant limitation is self-selection bias. Participants are drawn from a population of people who have already purchased an astrology report. They are, by definition, more interested in astrology than the general population and may be more susceptible to confirmation effects. This limitation is inherent to Phase 1 of the study and will be acknowledged in all published findings.

Phase 2, which will recruit external participants who have not purchased a Beaufort Intelligence report, is designed to address this limitation. External recruitment may use academic participant pools, social media recruitment, or panel services, and will require establishing credibility through the Phase 1 publication and, ideally, collaboration with an academic researcher. The results from Phase 1 and Phase 2 will be reported separately and compared.

A second limitation is the reading format. The study uses short readings of one to three sentences, extracted from longer report sections. A short reading necessarily strips context, and it is possible that the analytical accuracy of a full report depends on cumulative context that a single extracted reading cannot convey. If the study produces null results, this limitation must be considered before concluding that the full reports lack discriminative value.

A third limitation is the matched-era control design. While matching generational planets between real and randomised charts prevents generational markers from being the sole discriminator, it also means the randomised readings are not fully random. They are astrologically structured readings generated from a different but internally coherent chart. If the study produces null results, it may mean that any internally coherent chart produces readings that feel personally accurate, which would be an important finding in itself, though one that undermines the specificity claim rather than the experiential value of the analysis.

The study will be pre-registered on the Open Science Framework before any data is collected. Pre-registration is the single most important methodological commitment in this study. It publicly records the research questions, hypotheses, study design, sample size, and analysis plan before the data exists, making it impossible to retroactively adjust the methodology to fit the results. Nosek and colleagues have argued that pre-registration is the most effective safeguard against the analytic flexibility that has driven the replication crisis in psychology.^[11]

The pre-registration will include the exact wording of the research questions, the primary and secondary hypotheses, the statistical tests and their parameters, the sample size target and stopping rules, and the criteria for interpreting results as positive, negative, or inconclusive. Any deviation from the pre-registered protocol during data collection or analysis will be documented and justified in the published report.

The published dataset will be made available on Zenodo or OSF in anonymised form, allowing independent reanalysis. The study protocol, consent forms, debrief materials, and analysis code will also be published. The objective is complete transparency: any researcher with access to the published materials should be able to evaluate every decision made during the study.

The following commitments are made on record before the study begins. They are not conditional on the results.

The willingness to submit one's own product to a rigorous, blind, pre-registered test is not standard practice in the astrology industry, or in most industries that make claims about personal insight. The commercial incentive is to avoid precisely this kind of scrutiny. Beaufort Intelligence takes the opposite position: if the methodology described in the methodology pages produces analysis that is genuinely more accurate than chance, a well-designed study should be able to detect it. If it does not, that is information worth having, and information worth publishing.

This study will not resolve the question of whether astrology works. A single study never resolves anything. What it will do is produce a dataset, collected under transparent and replicable conditions, that contributes to the evidence base rather than the opinion base. The history of astrology research is littered with studies designed to confirm a prior commitment, either for or against. This study is designed to find out.

The protocol is published here, in full, before any data has been collected. It will be pre-registered on the Open Science Framework. The results will be published regardless of outcome. Anyone who believes the design is flawed is invited to say so before the study begins, while the methodology can still be improved. That is how research is supposed to work.