Yes, it does get a little muddled. Let’s give this a try, and then maybe you can respond for additional clarification if needed.
Early studies on human variation, prior to the human genome project (HGP) were restricted to working with alleles of single “genes” (in reality, generally short stretches of DNA that included a gene but also some DNA around it). These studies depended on the researchers actually going out and sequencing a large number of people for this specific gene, and then making sense of the allele diversity they found for that region (by modelling using mutation frequency, etc). These are not PSMC methods, but earlier coalescent-based methods.
For example, this early paper looks at a few such genes for which data was available at the time and concludes this (from the abstract, my emphases):
“Genetic variation at most loci examined in human populations indicates that the (effective) population size has been approximately 10(4) for the past 1 Myr and that individuals have been genetically united rather tightly. Also suggested is that the population size has never dropped to a few individuals, even in a single generation. These impose important requirements for the hypotheses for the origin of modern humans: a relatively large population size and frequent migration if populations were geographically subdivided. Any hypothesis that assumes a small number of founding individuals throughout the late Pleistocene can be rejected.”
Later pre-HGP papers were in agreement with these results. For example, this paper looked at another gene (the PHDA1 gene), and reports a human effective population size of ~18,000.
Another paper from this timeframe looked at allelic diversity of the beta-globin gene and found it to indicate an ancestral effective population size of ~11,000, and conclude that “There is no evidence for an exponential expansion out of a bottlenecked founding population, and an effective population size of approximately 10,000 has been maintained.” They also state that the allelic diversity they are working with cannot be explained by recent population expansion - the alleles are too old to be that recent. (This also fits with the genome-wide allele frequency data we see later from the HGP.)
It is in this timeframe that the Alu paper is also published. It looks at allelic diversity of a different kind. Alu elements are transposons - mobile DNA - and they can generate “alleles” where they insert. Generally, if an Alu is present, that’s an allele, compared to when an Alu is absent (the alternative allele). This paper is also nice because it does not depend on a forward nucleotide substitution rate - i.e. the DNA mutation rate, since Alu alleles are not produced by nucleotide substitutions. This paper concludes that the human effective population size is ~18,000. They also state (my emphases):
“The disagreement between the two figures suggests a mild hourglass constriction of human
effective size during the last interglacial since 6000 is very different from 18,000. On the other hand our results also deny the hypothesis that there was a severe hourglass contraction in the number of our ancestors in the late middle and upper Pleistocene. If humans were descended from some small group of survivors of a catastrophic loss of population, then the distribution of ascertained Alu polymorphisms would show a pre- ponderance of high frequency insertions (unpublished simulation results). Instead the suggestion is that our ancestors were not part of a world network of gene flow among archaic human populations but were instead effectively a separate species with effective size of 10,000-20,000 throughout the Pleistocene.”
From here, we start to get into what are really HGP papers but are focused studies on small DNA regions, rather than genome-wide variation. These are still not PSMC studies. For example, this paper looks at a small section of an autosomal chromosome (chromosome 22). They conclude (my emphases):
"The comparable value in non- Africans to that in Africans indicates no severe bottleneck during the evolution of modern non-Africans; however, the possibility of a mild bottleneck cannot be excluded because non-Africans showed considerably fewer variants than Africans. The present and two previous large data sets all show a strong excess of low frequency variants in comparison to that expected from an equilibrium population, indicating a relatively recent population expansion. The mutation rate was estimated to be 1.15 10 9 per nucleotide per year. Estimates of the long-term effective population size Ne by various statistical methods were similar to those in other studies. "
A second paper of this type looked at a region of chromosome 1. They also do a variety of estimates of population size for this region, and they conclude the following (my emphases):
An average estimate of ∼12,600 for the long-term effective population size was obtained using various methods; the estimate was not far from the commonly used value of 10,000. Fu and Li’s tests rejected the assumption of an equilibrium neutral Wright-Fisher population, largely owing to the high proportion of low-frequency variants. The age of the most recent common ancestor of the sequences in our sample was estimated to be more than 1 Myr. Allowing for some unrealistic assumptions in the model, this estimate would still suggest an age of more than 500,000 years, providing further evidence for a genetic history of humans much more ancient than the emergence of modern humans. The fact that many unique variants exist in Europe and Asia also suggests a fairly long genetic history outside of Africa and argues against a complete replacement of all indigenous populations in Europe and Asia by a small Africa stock. Moreover, the ancient genetic history of humans indicates no severe bottleneck during the evolution of humans in the last half million years; otherwise, much of the ancient genetic history would have been lost during a severe bottleneck.
In other words, the alleles we see in the present day cannot be explained as arising after a severe bottleneck in the last 500,000 years.
From here, we’re on to the HGP papers and later the 1000 genomes papers as they extend this sort of thing to the genome as a whole, show the allele frequency spectrum for a much, much larger dataset, and now we start seeing PSMC analyses included. There’s a lot to summarize in those papers, but the take-home message is those papers support the same conclusions as the previous work, but now using a massive data set. No one looked at the HGP/ 1000 genomes work and said it’s time to revisit the previous conclusion that a sharp bottleneck had been ruled out. On the contrary - the HGP/1000 genomes papers provide additional evidence that the prior work was solid.
So, there’s a full treatment of what is glossed as a few sentences in Adam and the Genome.
I’ll cover linkage disequilibrium (LD) (which is independent of the nucleotide substitution rate) and the single-genome PSMC approaches in my upcoming replies to Richard. Hopefully this gets you (and everyone else) up to speed thus far. Let me know if you’d like clarification on any of the above.