How We Use ICP-OES Results Of Unknown Accuracy And Precision

[Dan_P]

How are you thinking now - did you not send in all samples to the same vendor at the same time?

Sincerely Lasse

Sending the samples at the “same time“ does not equal “controlled conditions”, right? Or have I had two glasses of wine too many to understand your question?

 

[Rick Mathew]

How are you thinking now - did you not send in all samples to the same vendor at the same time?

Sincerely Lasse

Actually in my sample sets this was correct for the most part. With the exception of one set that I held back and kept in cold storage for two additional days to have a look at the time variable. All of my other sets were sent on the same day. They were all sent in separate shipping boxes like a single analysis. For one series this would be 3 sample sets to each vendor. Even though the samples were sent at the same time they did not arrive at the same time. Some arrive 1-2 days later and one 5 days later...USPS at their best! There is also the variable of how the vendors que the samples for testing as well as any backlog they might have or maintenance and calibration times... etc. I say all of this to say The sample time to measurement time at all the vendors is for sure a variable. So how samples behave in storage over time becomes a question...IMO

Rick

 

[Rick Mathew]

Sending the samples at the “same time“ does not equal “controlled conditions”, right? Or have I had two glasses of wine too many to understand your question?

Could it be that both are true!

 

[taricha]

Thanks, taricha.

Perhaps that points up limitations to the interpretations (presumably because the total number of samples was not really very large), since the ICP-A result from the reproducibility test said it was useless while the result from the precision test indicated it was pretty good. And it's not apparently a difference of reproducible vs precision, but rather a difference that a big outlier fell in one test and not in the other.

It's a bit of a challenge describing these things.
I think of it more that analyzing performance from one angle and method didn't reveal any great issues, but analyzing it from another perspective did.
The issues are real, just that one set of statistical tools doesn't reveal it.

The sample size is a limitation.
For instance, Chart 3 was generated from 12 separate data sets, that had an average of 8 ICP tests in each. All of them 2-(or more) way comparisons across vendor, time, stock addition, dilution...
And yet I still have no idea what to say about ICP measurements of Titanium between 1-100ppb. Data just isn't there to say much.

But most of all, it makes me really feel bad for those trying to decide what a received ICP result means and what to do off of just N=1 from a vendor.

 

[Lasse]

Sending the samples at the “same time“ does not equal “controlled conditions”, right? Or have I had two glasses of wine too many to understand your question?

Actually in my sample sets this was correct for the most part. With the exception of one set that I held back and kept in cold storage for two additional days to have a look at the time variable. All of my other sets were sent on the same day. They were all sent in separate shipping boxes like a single analysis. For one series this would be 3 sample sets to each vendor. Even though the samples were sent at the same time they did not arrive at the same time. Some arrive 1-2 days later and one 5 days later...USPS at their best! There is also the variable of how the vendors que the samples for testing as well as any backlog they might have or maintenance and calibration times... etc. I say all of this to say The sample time to measurement time at all the vendors is for sure a variable. So how samples behave in storage over time becomes a question...IMO

Rick

What i understand - your aim with a part of this study was not to compare results between different labs but to compare each labs ability to withhold repeatability between their analyse of the same or nearly the same sample. If each vendor get their sample at the same time and with the same shipment - the transport factor is zero for comparison between result from the same vendor and unknown between the vendors IMO - if I have not missunderstand your methodical approach

Sincerely Lasse

 

[Rick Mathew]

What i understand - your aim with a part of this study was not to compare results between different labs but to compare each labs ability to withhold repeatability between their analyse of the same or nearly the same sample. If each vendor get their sample at the same time and with the same shipment - the transport factor is zero for comparison between result from the same vendor and unknown between the vendors IMO - if I have not missunderstand your methodical approach

Sincerely Lasse

Your understanding of the study is correct we sent each vendor 3 or 4 of the same sample, but the samples were sent in individual shipping boxes not in the same box so the vendor did not necessarily get the samples at the same time. Although the samples were sent on the same day to a vendor from the same post they did not arrive at the vendor's location for testing on the same day. As I mentioned some arrived up to 5 days later. Why this is the case is unknown to me.

For example I sent 4 identical samples to Triton on Aug 23rd. All shipped on the same day but in individual boxes to represent a single analysis. 3 of the arrived on or before the 28th with a test date of the 28th. The 4th sample was reported by Triton as not having arrived. It finally arrived on or before Sept 1st and was recorded as tested on the 1st. The same 4 samples were sent to Faun Marin on the 23rd and all arrived on the 31st...However the the 9/27 sample set sent to Fauna Marin had arrival / test days 2-4 days apart having all been shipped on the same day.

This variability was not unexpected and in actuality represents what happens to samples in general that are sent to ICP vendors which was why we held back one set on purpose to "simulate' this variable.

I hope this helps

 

[Lasse]

In this case - It could be a good idea compare - as an example - Tritons 3 samples (arriving the same day) with the one arrived later and in FM case - IMO - you have an excellent opportunity to show if transport time matter or not - at least at FM.

Sincerely Lasse

 

[Randy Holmes-Farley]

But most of all, it makes me really feel bad for those trying to decide what a received ICP result means and what to do off of just N=1 from a vendor.

I agree. In that regard, its a pretty disappointing result for those wanting to use it with great accuracy.

It also serves to bolster my concerns about one of the vendors, which I do not recommend to folks.

 

[taricha]

In this case - It could be a good idea compare - as an example - Tritons 3 samples (arriving the same day) with the one arrived later and in FM case - IMO - you have an excellent opportunity to show if transport time matter or not - at least at FM.

almost....
except there's the other side too. There's a shift in the sample over time, and there's time-variation in the ICP-measurement process itself. Cleaning, recalibration, was it 1st sample after cleaning or last sample before next cleaning etc.

some cases are clearly sample changing, others look more like variation on the ICP lab side.
see this chart 1B from Fritz data.

now, obviously the same sample held for a month did not simultaneously increase concentration by 5%, decrease by 2%, and decrease by 10+%. This chart looks to me more like the shift in ICP machine recalibration was larger than the shift in these elements in the sample itself. So it actually makes conclusions about change in those elements in the stored sample impossible - except that it seems smaller than the variation on the ICP lab side.

 

[Rick Mathew]

In this case - It could be a good idea compare - as an example - Tritons 3 samples (arriving the same day) with the one arrived later and in FM case - IMO - you have an excellent opportunity to show if transport time matter or not - at least at FM.

Sincerely Lasse

Lasse: Not to add to the confusion....Tagging on to @taricha response.

In the sample set sent on 9/25/21 we set up an experiment to look at this very question. As in many experiment what we expected to happen did not. Two vendors were involved. The long and short of it was that the variability of the first samples to arrive at both vendors was too high as to be able to differentiate it from the variability of the second set of samples stored for 4 additional days...This supports @taricha conclusion on the FRITZ data which had a one month storage time....

 

[taricha]

So it actually makes conclusions about change in those elements in the stored sample impossible - except that it seems smaller than the variation on the ICP lab side.

Oh, last tidbit. Fritz's test kits for Ca and Mg showed them unchanged ~410 and ~1350 one month apart. Consistent with the idea that it's the ICP test that varied, and not the sample (to any detectable degree).

 

[taricha]

I agree. In that regard, its a pretty disappointing result for those wanting to use it with great accuracy.

Frustrating that some of the elements we'd really like to know are the low concentration, bio-active, rapidly depleted ones. And those are some that are the hardest for ICP for mostly the exact same reasons that we'd like to track them.

It also serves to bolster my concerns about one of the vendors, which I do not recommend to folks.

Yeah, I think in discussions Dan pointed out that many people already felt like that vendor generated poor results - just without much data to support that impression.
The data makes it a bit clearer.

 

[Roberto Denadai]

I'm so glad I never wasted my money on ICP

 

[Lasse]

I'm so glad I never wasted my money on ICP

On the contrary, I am glad that I have invested in ICP tests - without these, this would not have been possible to build up and maintain this for 6 years.


Sincerely Lasse

 

[taricha]

I'm so glad I never wasted my money on ICP

Yeah, that's not really a conclusion you can get to from our data.

Only that ICP measurements have a certain amount of real-world uncertainty - and you should make decisions with the size of the uncertainty in mind. Like any other measurement method.

Most vendors could distinguish zero from 10ppb for most of the trace elements. Some vendors could consistently distinguish zero from 1ppb for a few trace elements.

Some of the major elements could be measured by some vendors with high precision within the range that hobbyists would prefer to target: Na, Ca, Br for instance.
Some other major elements have larger uncertainties, and an ICP result showing slightly higher or lower than desired values shouldn't be taken as definitive.

 

[taricha]

@Lasse 's tank would be even better if he trusted hanna P measures over ICP
(but we'll agree to disagree )

 

[Lasse]

@Lasse 's tank would be even better if he trusted hanna P measures over ICP
(but we'll agree to disagree )

Since I start to filtrate the sample before measurements with Hanna (0.2 µ) - they are mostly the same. My aquarium have a lot of tiny particles and it seem that my Hanna have shown to much PO4 because of interreferences from particles. Only a small hint that you can test

Sincerely Lasse

 

[Reefahholic]

Interesting article. Thanks for sharing!

 

[jda]

Following.

I will also say that Lasse is an expert aquarist that could have probably had success sending in old water that boiled pasta for ICP. He is smart and experienced and could identify and withstand any type of garbage result.

My worry is for those who are newer or less experienced looking to trust something like this as an absolute, so well-done to express that this is not a good idea without more digging. I have seen too many ICP tests cause harm in various ways. ICP is just a tool and it needs to be used correctly.

 

[Roberto Denadai]

On the contrary, I am glad that I have invested in ICP tests - without these, this would not have been possible to build up and maintain this for 6 years.


Sincerely Lasse

You definitely don't need ICP to keep easy corals like yours.

I´m keeping all kinds of corals and some SPS tanks for 22 years without ICP. I doubt that ICP can help my tank look better.

Best Regards