Yeah, it’s not fair for other labs. You can’t do an experiment on a few cars and label all cars defective because a few of them didn’t perform well.
HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??
- Thread starter Rick Mathew
- Start date
- Tagged users None
Yeah, it’s not fair for other labs. You can’t do an experiment on a few cars and label all cars defective because a few of them didn’t perform well.
OP
OP
Rick Mathew
Valuable Member
Interesting point, but I would ask how would one know which ones are defective and which ones are not without a test drive? And on the drive you would need to have some clear criterion for making the judgement...some standard to evaluate their performance...Round Robin type experiments are standard practice in many industries and in some cases may be required for certification...The same sets of samples are sent to multiple labs that measure on different equipment and the results are evaluated as to their performance.
Last edited:
areefer01
2500 Club Member
No disagreement but maybe the conversation has shifted now and it shouldn't have. I was merely suggesting if a hobbyist is to use ICP then they should use the same one and prepare the sample consistently the same for ideal results. Basically control what they can before submitting. Then the results will be consistent assuming shipping and lab performance is consistent on their side.
Did not mean to take it outside of that as I was interpreting it as chasing numbers or asking which one is correct or better. Sorry about that.
OP
OP
Rick Mathew
Valuable Member
Actually this is a very important point you raised, so no apology necessary...Your point about using the same vendor is very valid and could possibly take a lot of confusion out of the equation...The question is now much variation is in the vendors results...The best way I can describe this is by example... if I send 10 samples of exactly the same sample to my vendor and they test them according to their test protocol how close would the measurements be? If they are very close then I can use the results to make decisions about my system, but if they are large then it makes it more difficult to know how your parameter is trending...This describes the precision of the test, but not the accuracy, that is another matter. ...hope this helps
Last edited:
areefer01
2500 Club Member
It does. And thank you for the reply. It then would raise the question for me is if the number is different because my system consumed or used it. Not that there was a testing error or question of precision. In my reply I am assuming (which very well could be the issue) that the vendor is doing everything correctly. Maybe this is the dangerous part for our hobby. Assuming vendor is doing things correctly (note I am not saying they are just putting fault on my assumptions).
OP
OP
Rick Mathew
Valuable Member
Most welcome
In the above experiment all 10 samples came from the tank at the same time so we take the system consumption out of the equation. The only thing we would be evaluation is the test protocol, therefore the only reason for different measurement results would be the test protocol itself...Also different results does not necessary mean the vendor is doing something incorrectly it is only an indication of the variability in the test ...EVERY test has variability, the question is how much.
MnFish1
10K Club member
Nice work. Couple questions - why 'Margin of error' as compared to Standard deviation? Did you see any trends, ie.. one company is not as good as the others (for ICP) - as a conclusion? Maybe I missed it in the extensive excellent write up - but I would be curious as to your opinion because - IMHO, people are going to continue to use ICP tests - and it would be nice to know if one or more underperform since as others have said - there are multiple chemicals for which there are no hobby tests available.
Do you think the 'average hobbyiest', ie. someone just starting out, would perform the tests as well as the three of you?
Do you think the 'average hobbyiest', ie. someone just starting out, would perform the tests as well as the three of you?
- Joined
- Sep 21, 2018
- Messages
- 7,571
- Reaction score
- 7,962
We (@Rick Mathew @taricha) discussed a similar question in the context of the bigger question of why vendor results disagree when testing the same sample. Machine differences was one factor but we also considered procedural differences, quality standard differences, machine drift between maintenance cycles, bad luck, etc. And then there is the potentially really big problem: variable sample handling. The sample quality is never controlled before testing and we do not know how our sample has changed before it is tested. Taken together, is it any wonder we are concerned about ICP test accuracy.
Our study showed for eight elements that ICP and colorimetric tests produce similar results. Considering what I said about the challenges facing ICP vendors, ICP results reported in our study for some of those eight elements are surprisingly good for 50 USD. In the end ICP data as noisy as it is, might be/probably is good enough to support reef aquarium management.
areefer01
2500 Club Member
Thank you. It makes sense but the part that keeps tripping me up is similar to what we see here and elsewhere regarding say the Trident. Some hobbyist report great results while others do not. Same piece of equipment but different hobbyist experiences. Is it a similar experience with ICP vendors?
As I reply I understand now that I'm changing the test which isn't right. I'm saying to myself the test would have been more valuable running the number of tests over time using the same vendor for accuracy and trend. Not a single sample to verify vendors or answer a different question.
Apologies.
- Joined
- Sep 21, 2018
- Messages
- 7,571
- Reaction score
- 7,962
You aren’t alone.
I only half listen to reports about tests being good or bad (including ICP) because the person reporting this never makes their judgement based on a controlled experiment or using a standard. Typically, the test is judged “good” if it agree‘s with an expectation or another unvalidated test result. The only news about a test I listen to is how they felt about using the methodology. And as you might guess, trust and a vendor’s credentials play no roll in judging the quality of a test result.
Pod_01
Valuable Member
In my opinion they should have send 5 or 10 samples to single ICP machine.
Again you will get the question which one…..
Right now part of the data I consider a partial screening exercise of ICP vendors.
Now we know they are different so who provides the best results for my $$$$. Or perhaps the question should be who provides best results and for what elements…
In the end I am good with what was done, lot of work went into it time and $$$.
I might have different interpretation, but I feel good with the ICP provider I use and this helped me feel comfortable with my choice. I am planning to change nothing in my reefing process, for now, like next 5 minutes…..
Few pretty pictures what this is all about at the end:
I do this hobby for the corals not the testing procedures, I wish the testing was transparent/not required even better.
I do get it others do this for the tech etc…
I just want pretty, colourful corals….
OP
OP
Rick Mathew
Valuable Member
Very Nice!!In my opinion they should have send 5 or 10 samples to single ICP machine.
Again you will get the question which one…..
Right now part of the data I consider a partial screening exercise of ICP vendors.
Now we know they are different so who provides the best results for my $$$$. Or perhaps the question should be who provides best results and for what elements…
In the end I am good with what was done, lot of work went into it time and $$$.
I might have different interpretation, but I feel good with the ICP provider I use and this helped me feel comfortable with my choice. I am planning to change nothing in my reefing process, for now, like next 5 minutes…..
Few pretty pictures what this is all about at the end:
I do this hobby for the corals not the testing procedures, I wish the testing was transparent/not required even better.
I do get it others do this for the tech etc…
I just want pretty, colourful corals….
OP
OP
Rick Mathew
Valuable Member
Thanks for your reply....It would actually be great if the vendors themselves would test the same sample 10 times and report the results so we could see the precision of their testing methods....Save us a lot of money a?In my opinion they should have send 5 or 10 samples to single ICP machine.
Again you will get the question which one…..
Right now part of the data I consider a partial screening exercise of ICP vendors.
Now we know they are different so who provides the best results for my $$$$. Or perhaps the question should be who provides best results and for what elements…
In the end I am good with what was done, lot of work went into it time and $$$.
I might have different interpretation, but I feel good with the ICP provider I use and this helped me feel comfortable with my choice. I am planning to change nothing in my reefing process, for now, like next 5 minutes…..
Few pretty pictures what this is all about at the end:
I do this hobby for the corals not the testing procedures, I wish the testing was transparent/not required even better.
I do get it others do this for the tech etc…
I just want pretty, colourful corals….
....When we set up the design of the experiment we attempted to maximize the results we could get with our budget...The NIST traceable standards cost a pretty penny but we felt it was necessary to get at least an indicator of accuracy...Thanks again for the replyBeautiful Coral!!
I agree, but that’s not what the thread title was about. The title was about Hobby grade test kits outperforming ICP measurements, not which labs are more accurate than others.
- Joined
- Sep 21, 2018
- Messages
- 7,571
- Reaction score
- 7,962
It was not the purpose but it was a tightly interwoven piece of information with the outcome of our work.I agree, but that’s not what the thread title was about. The title was about Hobby grade test kits outperforming ICP measurements, not which labs are more accurate than others.
We discussed quite a bit about the best way to present the ICP data to the R2R readers and decided reporting all ICP data rather than an average. We felt the full data set might help folks get a better perspective about how good and how not so good hobby ICP testing is. What we haven’t shared yet is the analysis and perspective about the other elements. A bunch more work is needed to pull together our various studies and other published work. Maybe in 2024.
Absolutely. It’s only noisy if you try to compare all the labs to each other. The same would happen if you compare all salt brands.
It helps folks understand how good/bad some labs may be. I do appreciate the data in that regard.
One thing I want to point out about Plasma Interfaces.
Fauna is using a dual side radial view for their seawater testing. OCEAMO is using a axial plasma view.
Let’s look at some differences:
An Axial-view system looks from end to end of the plasma’s entire axis. It observes everything going on in the excitation channel.
The Radial-view system looks across the plasma. It sees only a narrow cross section of light rather than the light from the whole length.
Having less light to process, a Radial system can’t match the “sensitivity” of a Axial system in detecting trace elements.
Radial can be more precise and require less maintenance, but you loose sensitivity.
Axial-view detection limits surpass Radial-view performance by a factor of 10. Performance of the new Radial machines have improved, but most labs aren’t running them. They still have the older machines. However, for those that maybe updated their machines, Axial detection limits are still “2-3x lower” than Radial and get you the “lowest detection limits” possible using a ICP-OES machine.
They do require more cleaning and are less stable, but if the operator keeps the machine clean it’s not a problem.
I believe this is why Fauna has
Switched to the Spectro green for seawater sampling. They have a lot of testing going on, and want to avoid having to clean the machines frequently, but they’ve sacrificed sensitivity. It’s great for them, but not so great for us.
Fauna is using a dual side radial view for their seawater testing. OCEAMO is using a axial plasma view.
Let’s look at some differences:
An Axial-view system looks from end to end of the plasma’s entire axis. It observes everything going on in the excitation channel.
The Radial-view system looks across the plasma. It sees only a narrow cross section of light rather than the light from the whole length.
Having less light to process, a Radial system can’t match the “sensitivity” of a Axial system in detecting trace elements.
Radial can be more precise and require less maintenance, but you loose sensitivity.
Axial-view detection limits surpass Radial-view performance by a factor of 10. Performance of the new Radial machines have improved, but most labs aren’t running them. They still have the older machines. However, for those that maybe updated their machines, Axial detection limits are still “2-3x lower” than Radial and get you the “lowest detection limits” possible using a ICP-OES machine.
They do require more cleaning and are less stable, but if the operator keeps the machine clean it’s not a problem.
I believe this is why Fauna has
Switched to the Spectro green for seawater sampling. They have a lot of testing going on, and want to avoid having to clean the machines frequently, but they’ve sacrificed sensitivity. It’s great for them, but not so great for us.
Similar threads
