HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

[taricha]

After a chat with @Christoph I wanted to talk about a point of seeming disagreement that really isn't, and clarify potential misunderstanding.
So let's say Christoph reports that Oceamo's error in major elements is less than 3% or something of the sort. Meanwhile, we report that they only detected 66% of the spiked Magnesium, those seem in conflict - but aren't actually...

A quick look at the numbers shows why: We're reporting error as a percent of the known spike, and the ICP company is talking about error as a % of the entire value.
The consensus Mg value between ICP and Chem testers was around 1350 ppm, we spiked this value with +100ppm or +7.41% of the total.

Oceamo detected 66% of the 7.41% so they missed 34% of a 7.41% spike = 2.52% miss between two measurements of Magnesium.

Statistically we can do the error propagation for subtraction, because measuring a spike requires subtracting two measurements that each have errors.

2.52% = sqrt( error1^2 + error2^2 )
Solving the above gives that error1 and error2 are an average of 1.78%

So a 1.78% error in Mg is expected to explain recovering 66% of our spike in Mg.

So if you want to raise your Mg by 100ppm, ICP must be very good to allow you to trend that difference and see most of it. As you can see from our results in the chart, titrations are quite good for this - as taking care with them makes them very repeatable.

 

[fr3n0z]

The air volume inside a syringe pulling up liquid reflects the air volume in the tip. How hard/how far you push the tip onto the syringe barrel can impact the air volume included in the "tip volume".

In general, I show how to use a syringe here:

Reef Chemistry Question of the Day #161 Using a Syringe

Reef Chemistry Question of the Day #160 I have just drawn tank water into a syringe with a tip (shown below, including a blowup of the barrel). The blue coloration shows where the fluid is. When I started to draw fluid in, the tip and syringe were empty and the plunger was all the way in...

www.reef2reef.com

yep i saw your post. Thing is i am trying to be as constant as possible and even then i find that sometimes liquid is 0.84 sometimes 0.83 or 0.85 and it seems weird. but probably is such a small amount that the slight/imprecettible difference in using the siringe it will change the end quantity.

 

[fr3n0z]

This is a good observation and it points to the bigger issue of why when we use hobby grade test we often get large variability. Most of us are not trained in good laboratory practices and these very small things (and there are many of them) add up to some significant variability..Errors generally don't compensate they accumulate. You might find the series (4) of articles titled "GETTING IT RIGHT" to be helpful ...here is a link to the article page where all 4 are located

Articles by Rick Mathew

www.reef2reef.com

cool man, that is was i was talking about. Even more ...thanks a lot!

 

[Rick Mathew]

cool man, that is was i was talking about. Even more ...thanks a lot!

most welcome ....any time

 

[Reefahholic]

I agree with that. High quality analytical testing is very expensive. To think that one can get very high quality data for dozens of different analytes for one low cost is asking a lot. As I've mentioned in other threads, I routinely outsource analytical testing of various sorts, and $100 for a single analyte in a single sample, even when submitting dozens of the same types of samples, is not unusual.

I think it’s a pretty good deal for what he offers. It took Andre and Christoph a long time to come to an agreement on this deal. I kept telling Christoph it would be worth it, because we have around 5K people that are switching over from ATI. Christoph is in Austria, and when you get into customs and all that junk, it’s getting really expensive to ship. Andre is loosing money to make this deal from what I understand. He sells the tests on his website, and they get shipped to you. You sample the water and they get sent back to his mailbox in Houston. He ships out on Monday and Friday evenings. Soon I think Wednesday evenings also so they will be going out 3x a week with very fast turn around. Currently it’s about 7-9 day’s turnaround.

 

[Reefahholic]

After a chat with @Christoph I wanted to talk about a point of seeming disagreement that really isn't, and clarify potential misunderstanding.
So let's say Christoph reports that Oceamo's error in major elements is less than 3% or something of the sort. Meanwhile, we report that they only detected 66% of the spiked Magnesium, those seem in conflict - but aren't actually...

A quick look at the numbers shows why: We're reporting error as a percent of the known spike, and the ICP company is talking about error as a % of the entire value.
The consensus Mg value between ICP and Chem testers was around 1350 ppm, we spiked this value with +100ppm or +7.41% of the total.

Oceamo detected 66% of the 7.41% so they missed 34% of a 7.41% spike = 2.52% miss between two measurements of Magnesium.

Statistically we can do the error propagation for subtraction, because measuring a spike requires subtracting two measurements that each have errors.

2.52% = sqrt( error1^2 + error2^2 )
Solving the above gives that error1 and error2 are an average of 1.78%

So a 1.78% error in Mg is expected to explain recovering 66% of our spike in Mg.

So if you want to raise your Mg by 100ppm, ICP must be very good to allow you to trend that difference and see most of it. As you can see from our results in the chart, titrations are quite good for this - as taking care with them makes them very repeatable.

I’m glad you were able to speak with Christoph. As you probably know now, he’s a very honest and trustworthy guy. Anybody that has spoken with him can see that straight away. I just like the guy and trust his work. Nothing is perfect, but I’ve been very happy with my results. Of course if I get a questionable value on a critical element, I’m not gonna make that correction without a second confirmation.

 

[BeanAnimal]

I think it’s a pretty good deal for what he offers. It took Andre and Christoph a long time to come to an agreement on this deal. I kept telling Christoph it would be worth it, because we have around 5K people that are switching over from ATI. Christoph is in Austria, and when you get into customs and all that junk, it’s getting really expensive to ship. Andre is loosing money to make this deal from what I understand. He sells the tests on his website, and they get shipped to you. You sample the water and they get sent back to his mailbox in Houston. He ships out on Monday and Friday evenings. Soon I think Wednesday evenings also so they will be going out 3x a week with very fast turn around. Currently it’s about 7-9 day’s turnaround.

A good deal in context to what? Not knocking him or any other vendor, but I think that results show that ICP is not nearly the panacea that people expect it or want to be.

My point remains the same. Given the result, it appears to be me that

A) these tests are no more accurate than any other test when it comes to what is available for hobby use.

B) Ignoring A and accuracy but given the varied results and what we know about how these tests are conducted and interpreted, how does one even begin to trust the precision (repeatability) from month to month, test to test.

C) for most the intended purpose is trace element dosing and given A or B - it is rather curious as to how one can precision dose trace elements of one can't precision measure them or even reliable (or cost effectively) trend what is there.

D) Given A, B and C - for those just looking for a baseline and trending - the test results indicate a home test will do the same for far cheaper. And you can cross check reagent batches in parallel when migrating to a new batch. Repeatability.

 

[ingchr1]

They can actually vary quite a lot...

Stability is important, but maybe not as important as it's made out to be for the average hobbiest then?

PO4 is really a subjective topic...

Indeed, as many other things seam to be in this hobby.

 

[Reefahholic]

A good deal in context to what? Not knocking him or any other vendor, but I think that results show that ICP is not nearly the panacea that people expect it or want to be.

Provide me a link or web address to another lab with his skill level that offers his services at the same price or cheaper.

how does one even begin to trust the precision (repeatability) from month to month, test to test.

Because I was looking at, and discussing the results of several analyses before I started sending out my own. Then I saw the results in my own system after I sent multiple ICP-MS. When I made corrections off his results the tank started to really take off. I haven’t received any results that I would consider abnormal or skewed considering how the tank is growing, what I’ve been dosing, trending, etc.

D) Given A, B and C - for those just looking for a baseline and trending - the test results indicate a home test will do the same for far cheaper.

For only 8 elements though. What about the other 40+ elements/pollutants/source water.?

 

[Reefahholic]

What would you say to this guy who used “inaccurate” ICP’s to correct his elements to help this animal go from the brink of death to thriving?

Or what about mine?

My colors are only getting better and better.

 

[Randy Holmes-Farley]

So let's say Christoph reports that Oceamo's error in major elements is less than 3% or something of the sort.

Is that a real number from Oceamo, or just an example? Does he show error info somewhere?

 

[taricha]

Is that a real number from Oceamo, or just an example? Does he show error info somewhere?

It's a claim I think they would make, but It's not a printed claim that I've seen. Just my characterization of how he expects his machines to perform on major elements based on discussions and day to day data like what was posted here:
https://www.reef2reef.com/threads/c...ms-seawater-measurements.997436/post-11594505

If they did make such a claim "< 3%" I'd find it plausible based on data like ours and Sanjay's.
If we run the same calculation as above that got 1.8% for Mg....
I get 3.0% for Ca and 0.7% for K.
We can also look at the percent off from the consensus (with the caveat that consensus may not be a great mark for the true value, but it's a decent ballpark of size of plausible errors.)
In our data Oceamo differed from the Chemical + ICP consensus values by....
Mg: 1.6% unspiked, 0.9% spiked
Ca: 0.5% unspiked, 1.5% spiked
K: 3.6% unspiked, 4.5% spiked

For comparison, in Sanjay's data, Oceamo differed from the consensus measures by...
Mg: 1.4%
Ca: 2.0%
K: 2.4%

So it'd be plausible, from what I've seen.

 

[Randy Holmes-Farley]

OK, thanks.

 

[Echale3]

Is that a real number from Oceamo, or just an example? Does he show error info somewhere?

In terms of lab analyses, error in terms of reported values versus true values are dependent on a variety of things. The sample matrix is one, as some elements/compounds affect the detection/recovery of other elements/compounds, which is why labs are supposed to run matrix spike/matrix spike duplicate samples. Then there's the signal to noise ratio of the detector being used -- the higher the S/N ratio, the higher the amount of element/compound needed to be present to provide a readable, reproducible signal above the detector noise floor. There are also other potential factors affecting precision and accuracy, things like non-linear detector response above or below the range of analyte levels, which is why some samples require dilution in order to get the amount of analyte present in the sample down into the proper detector response range. Depending on what you're diluting with and how accurate your dilution measurements are, you can induce mechanical error, or, in some cases, you can change the bias levels in terms of analyte response, which affects the reported analyte value.

I could go on about this for a while, but this gives you a pretty good idea of the limitations in terms of accuracy and precision for lab results.

The other thing to remember is that many labs are geared towards high sample throughput in order to keep analytical prices low and their QA/QC often suffers as a result. I run into this all the time, unfortunately.

The only good way I know for you to be able to determine for yourself what level of comfort you have with a reported value is to review all the batch QC data and make sure that the initial and continuing calibration analyses are in spec, the blanks are truly blank, the matrix spike/matrix spike duplicates give a clear indication of the type (low, none, or high) biasing to the reported results, the results of any dilutions required to get the analytical range within the high/low values for the calibration, etc., and make your judgement call based on that.

Given what I've seen in recent years, I'm not always particularly comfortable with the numbers I get back because I look at the lab error codes and don't always like what I see. Then again, I ran a GC/MS lab for a number of years, trained in ICP and Flame AA metals analysis, and was a lab data validation specialist for a state environmental regulatory agency in their solid and hazardous waste division before going back into private industry doing consulting, so I'm all about data quality.

 

[Daniel@R2R]

This is absolutely awesome!! Thanks to the authors for their contribution of outstanding work to the reef community! We appreciate you!

 

[Rick Mathew]

This is absolutely awesome!! Thanks to9 the authors for their contribution of outstanding work to the reef community! We appreciate you!

Most welcome....Glad to help...

 

[BeanAnimal]

Provide me a link or web address to another lab with his skill level that offers his services at the same price or cheaper.

I would likely choose the ICP-MS if I were forced to make a choice...

But a broken compass is a broken compass, regardless of the cost. It is not a useful tool if you need to find your way, even if it is broken differently than the competitors compass. So why buy one at all?

For only 8 elements though. What about the other 40+ elements/pollutants/source water.?

If the 8 elements can't be measured with certainty, what makes you think that the other 40 can be?

 

[Reefahholic]

I would likely choose the ICP-MS if I were forced to make a choice...

But a broken compass is a broken compass, regardless of the cost. It is not a useful tool if you need to find your way, even if it is broken differently than the competitors compass. So why buy one at all?

If the 8 elements can't be measured with certainty, what makes you think that the other 40 can be?

I think that is a very smart choice. I’m definitely a huge fan of ICP-MS if you haven’t noticed. HaHa.

I’ve seen enough in my own system and others to know that the compass isn’t broken. Maybe not for every lab, but for the labs I’ve been using personally. There was one time where I did see ATI come back with high copper for several of us (5-10 people). It was like 10-12 ug/L. Normal would be below 1-2 ug/L. I target about 0.7. We all knew that it was most likely inaccurate, because none of us had done anything different. Another time I had Zinc come back around 20 ug/L and I target about 5. I sent a Triton off the next day (fast turnaround) to verify, and it was actually accurate. So I stopped Zinc for a while. Zinc can have a very negative effect at higher levels just like Copper. These are the only two incidents I can remember where something was “significantly” off, and only the Copper incident turned out to be inaccurate, because the next analysis came back normal. I’ve seen some low salinity readings, but I believe ICP has the more accurate data. It comes out pretty close to my Tropic Marin Hydrometer. Recently I had a incident with elevated Vanadium, and Christoph reached out to me asking if I adding a lot of ceramic frag plugs. I said…actually, I did add quite a few over the last 2-3 months. I put them in the sump to soak, but after he mention that could be the problem, I removed all of them. Oops! I’ll be able to see the results of that on this next analysis.

In some of these cases (which are very few) if you know your system you’ll likely already have a good idea how accurate the anomaly is (if there even is one). At that point you can make a educated decision to act or do nothing. Or maybe get a second conformation from a different lab, and most will retest the sample water if you call or email them.

The point I’m trying to make is that this is fairly rare, and there are ways around it. What if you didn’t have the data at all? You can’t make any decisions. Keep in mind, You DON’T Have to ACT unless YOU want to. I find the majority of the time I’m getting very accurate results, and I haven’t got any crazy data from OCEAMO yet.

I’ve helped countless people recover their systems, and that wouldn’t have been possible without having good ICP data. I’m not saying it can’t be off by 10-15ppm. Sure it can, but most reefers know to question critical elements and not to dose them if it could potentially crash their system. For main elements ICP has been very accurate for me, and for trace elements, MS has been fantastic. Typical detection limits are 0.05 ug/L or better for most elements. That is really impressive. We’re taking 10,000X more sensitivity. Most of my trace metal targets are 0.3 ug/L or higher, and the MS has no problem getting down there. With Uranium it’s able to get down to 0.001 ug/L which is mind blowing. That is 1 ng/L (Nanogram) I believe!

For me it’s just as much about the pollution in a tank as it is for the actual elements being dosed. We can’t maintain a thriving reef tank with pollutants. I’ve seen so many tanks with problems that stem from pollution. Being able to see their RO water has been a game changer in many cases. I’ve seen zinc and other crap coming through their filters at very high levels a number of times which turned out to be the primary problem. I’ve seen heaters, wavemakers, and frag racks breaking down. I’ve seen Tin flaking off cabinet hinges. If you were not sending ICP, none of these issue would have been known or addressed. For pollutants accuracy doesn’t matter as much, because we’re not dosing them. We just need to know if they’re present. Having the ability to see Tungsten, Neodymium, Copper, Antimony, Zinc, Vanadium, etc…is a huge help in diagnosing some serious or tank crashes.

Look, I only continue to write to you guys, because having more data to act on is super important. I get it, the data needs to be accurate, but from what I’ve seen it’s good enough, and better than not having data at all. We have tools that take all this data, and do these calculations for you, or give you guidance about what to do which is extremely helpful for newer reefers, and honestly even season veterans. It makes it easy for everybody. Even the mathematicians avoid using their brian’s. You can see your exact daily dose, daily elemental dose, your resulting multiplier (how many times you’ve increased each ultra trace element). It just makes life much easier, and quite frankly reefing becomes more enjoyable for everybody.

The tool may tell you to add 1X your “daily standard dose” which may be something like .01 - 1 ug/L starting out depending on your system volume. The precision we have with ICP-MS is really amazing, and I want all you guys to experience that. I’ve mentioned before that you can literally see the elements coming up/down by 0.1-0.2 ug/L as you’d expect to see (even less, but that is just an example), and colors becoming more intense reflecting the changes. This is verification that the corals are responding accordingly to dose corrections, and good data was received which is echoed on subsequent analysis. I’ve been playing with ICP for like maybe 3-4 years, and I’ve never crashed a tank behind it. Most labs are becoming competitive with one another, and before you know it they will all be filtering samples, using nutrient stabilizer’s.

I believe most labs will be using MS in the future once they see the value. I believe they already see it now, but the machines are expensive, and maybe they don’t have the money to upgrade so they continue to market and push their OES machines. I don’t want to knock OES too much, because for the majority of elements I do find they are accurate depending on the lab.

I’m not trying to convince you to adopt my method, although you’re more than welcome to join in, and I’ll help you to get started with as much as I can, but I do want to persuade you to get a quality ICP and start looking at the data. Continue to compare with your home test kits, and if you see it’s trustworthy over time, start making corrections on some of the elements like Strontium, Potassium, & Boron that maybe you haven’t corrected before. Then maybe start correcting some of the Halogens like Iodine, Bromide, Fluoride, etc. to see what you observe. At the same time keep an eye out for corroding equipment and source water. It’s just smart reefing.

 

[Pod_01]

If the 8 elements can't be measured with certainty, what makes you think that the other 40 can be?

Looking at the summary I am not sure if anything is broken. The ICP and the hand test were inline at picking up the portion of the spike. I am actually floored that not many overestimated (odd really). I would expect to have few more over the 100%.

The chart below consolidates all the above data into the averages of both the ICP testers and the hobby grade testers.

On some of the elements, both the hobby test kits and the ICP vendor averages seem generally good at capturing the amount spiked

Iron stands out but results are not unexpected (reasonable explanation was provided), Mg is interesting and perhaps something ICP companies should look into. But, does it make a difference if Mg is 100 less or more? I have not dosed Mg in over 4 years….
For phosphorus I wish there was accurate tool (measuring ortho PO4 is only part of the answer). The rest seems reasonable to me.

My take away is that affordable ICP and hobby test kits can’t measure the absolute but are good at relative values. In my opinion ICP gives you better overall view of the tank.
I think of ICP as the 80% solution. Do I really want to pay to get the last 20%???

 

[Reefahholic]

It's a claim I think they would make, but It's not a printed claim that I've seen. Just my characterization of how he expects his machines to perform on major elements based on discussions and day to day data like what was posted here:
https://www.reef2reef.com/threads/c...ms-seawater-measurements.997436/post-11594505

If they did make such a claim "< 3%" I'd find it plausible based on data like ours and Sanjay's.
If we run the same calculation as above that got 1.8% for Mg....
I get 3.0% for Ca and 0.7% for K.
We can also look at the percent off from the consensus (with the caveat that consensus may not be a great mark for the true value, but it's a decent ballpark of size of plausible errors.)
In our data Oceamo differed from the Chemical + ICP consensus values by....
Mg: 1.6% unspiked, 0.9% spiked
Ca: 0.5% unspiked, 1.5% spiked
K: 3.6% unspiked, 4.5% spiked

For comparison, in Sanjay's data, Oceamo differed from the consensus measures by...
Mg: 1.4%
Ca: 2.0%
K: 2.4%

So it'd be plausible, from what I've seen.

I’d say under 5% is still pretty acceptable wouldn’t you?