NZ Skeptics Articles

How do we really know?

Margaret Coe - 9 June 2025

How does a laboratory person know that they can trust the results being produced by an analysis? They use a validated method. I will describe the steps normally taken to validate an analytical method.

The most common methods of analysis in the world today are chromatography methods1. I won’t go into the details about the analysis, but what you need to know is that it involves a separation system with a detector. There are many types of separation systems and also many types of detectors. They all produce pictures of mixtures that look like a series of mountain peaks:

Detector response is the y axis and time is the x axis.

So, to validate a chromatographic method, you need to prove 2 main things, although others will crop up as we go along. These 2 main things are precision and accuracy. These words are often used interchangeably in our everyday speech, but in a lab setting, they have very different meanings. Precision is how close together are results using a method and accuracy is how close those results are to the correct answer. This is shown in these diagrams:

Good precision but poor accuracy

Good accuracy but poor precision

Good precision and good accuracy

To demonstrate precision, you would inject the same sample into the chromatographic system many times, typically at least 20x on at least 2 days.

If the precision is not good, you first need to figure out why and eliminate the problem. Usually, you won’t be able to inject samples directly, but must first extract the analytes of interest. So, poor precision could be the result of an inappropriate extraction method. This could be an extraction method that extracts things other than the analyte of interest. This is usually the case when dealing with samples such as blood or plasma or even relatively simple matrices like water or a solvent. The extra things that you extract might elute near the same time as the analyte of interest, interfering with its peak. If this is the case, you will need to change the chromatographic conditions until you have the analyte of interest all by itself. This can be an interesting exercise, taking lots of time. If you have the exact same amount of interfering substance eluting at the exact same time, every time, you can put up with it, but this is very much a non-ideal situation, as we will soon see. This is also a very unusual situation.

The acceptable precision will depend on the analysis. Looking at very small amounts will usually result in a larger value for precision. This might still be OK, if you are looking at the trend of the analyte, but it is far from ideal, as you would need very large differences to be sure they were real.

So, let’s assume that we have got an acceptable precision, with no interferences in the chromatogram. Next, we need to demonstrate good accuracy. For that, we will need to have a standard. We could buy the analyte of interest from our friendly neighbourhood chemical store, and it may come with a certificate of analysis. If that is traceable to something like ASTM or NIST2, that’s great. Usually, they aren’t, so you might feel the need to buy a true standard from one of those organisations, for mega bucks. Fairly often, the analytes we are looking for are not available commercially. Then, we ask our friends in the Chemistry department if they could make one. This is usually a good project for their students. Sometimes they won’t or can’t make a standard, though. So, what to do?

Well, in that situation, which is not too unusual, we can use something of similar structure to act as a standard. We hope this has a similar response from the detector we are using. If our detector is a mass spectrometer (MS), then all is not lost. We can show that our analyte and the substitute standard produce similar mass spectra and fragments. So, it will be a reasonable assumption that it will be an OK standard.

If you are in the lucky situation that you have a mass spectrometer and a commercially available standard, you might also be able to buy a ‘labelled’ one, with some deuteriums substituted for the hydrogens in the analyte. This will have almost exactly the same retention time as the analyte, and will behave in exactly the way it would in the MS, but with its mass changed by 2 or 3 amu. Then, you can use this deuterated standard as an internal standard, which will help both precision and accuracy.

Once we have a standard, we need to generate a calibration curve by analysing a series of concentrations covering the range we expect to find, in the extraction solvent. We’d really like this to be linear, but it could be any of those shown, and all are usable, as long as the response of your analyte is close to those in the standard. These all show response on the y axis and amount on the x axis.

Linear through zero

Linear, showing a response with nothing injected; not ideal. Remember that interference?

Linear showing no response with something injected. This helps with LOD.

Best to use the first part of these; dilute samples to be here.

Criteria for acceptance

The LOD is the Limit of Detection of the method. If the linearity graph intersects the x axis, then the LOD is fairly simple. Otherwise, we need to analyse ever decreasing amounts of standard until we just barely have a signal. Then the LOD is taken as at least 3x that amount, often 10x that amount, depending on the application.

This is when you may realise that the method doesn’t have a low enough LOD to be useful, so it’s back to the drawing board to figure out another way to analyse it. With today’s detectors this is very rarely an issue, but it does sometimes happen. Then, you probably have to compromise and report a ‘less than’ figure, which we’d rather avoid.

So, you’re probably wondering how long this all takes. With a straightforward method, with no interferences, it can be done in a week, if everything goes perfectly. However, I have had methods take 4 months of continuous work to develop. What a wonderful feeling when it finally is right.

  1. For those keen etymologists reading this, yes, the name comes from the words writing with colours. This is because the technique was first used to separate the components of plant leaves, and different colours of chlorophyll were observed.

  2. ASTM = American Society for Testing and Materials, now known as ASTM International; NIST = National Institute of Standards and Technology