HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

[Reefahholic]

Cool. I'd like to see that data in a spreadsheet with a bunch of examples.

I have a recent spreadsheet from a new guy. Keep in mind that he just started so he’s not a pro. However, you can clearly see that from ICP #1 to ICP #10 that his elements are indeed trending in the right direction, and the chemistry is becoming stronger. He’s at the point where these elements will begin to stabilize much more making dosing easier. One thing we need to consider is that when many of these elements are depleted there are two things that usually happen as you dial them in.

1. The consumption/growth increases.

2. It can take some time for elements to saturate enough to become stable.

As a result, it’s kind of a “cat and mouse” game… dosing > increasing the dose > more consumption/growth > increasing the dose > elements start to saturate and stabilize a bit.

FYI…First test is ATI OES….the next 9 are ICP-MS

And here’s his current consumption. As you can see things have taken off quite a bit.

 

[Reefahholic]

I can also tests for strontium but the tests is not good it has a high degree of variability 30-40% and I think Salifert is discontinuing their test..

I had a feeling that’s why you guys didn’t include Sr. I tried that test a while back, and did it more than once only to become frustrated with the results. Thank you for being honest.

 

[Rick Mathew]

I had a feeling that’s why you guys didn’t include Sr. I tried that test a while back, and did it more than once only to become frustrated with the results. Thank you for being honest.

I actually used up an entire test kit running against known solutions and trying to refine the test...NO joy!! The calcium interference issue caused lots of problems...I think HACH makes one...might try it later and see how it does.

 

[BeanAnimal]

I have a recent spreadsheet from a new guy. Keep in mind that he just started so he’s not a pro. However, you can clearly see that from ICP #1 to ICP #10 that his elements are indeed trending in the right direction, and the chemistry is becoming stronger. He’s at the point where these elements will begin to stabilize much more making dosing easier. One thing we need to consider is that when many of these elements are depleted there are two things that usually happen as you dial them in.

1. The consumption/growth increases.

2. It can take some time for elements to saturate enough to become stable.

As a result, it’s kind of a “cat and mouse” game… dosing > increasing the dose > more consumption/growth > increasing the dose > elements start to saturate and stabilize a bit.

FYI…First test is ATI OES….the next 9 are ICP-MS



And here’s his current consumption. As you can see things have taken off quite a bit.

or

3. There is that much imprecision in the tests.

Sometimes the simplest answers are the most probable.

 

[BeanAnimal]

Don’t get me wrong - I want ICP and a results to dosing interpreter to be the answer… I am jus not seeing the precision to make the leap.

 

[Reefahholic]

or

3. There is that much imprecision in the tests.

Sometimes the simplest answers are the most probable.

If I hadn’t of previously looked at about 100K ICP analyses, I might believe that.

With imprecision you don’t get crazy consumption and growth. There’s his current consumption. It’s quite a long way from where he started.

 

[Thales]

Ok, so you’re admitting that how you’re current way of testing is no better. So the question is why such strong opinions in regards to ICP results?

I think you are attributing stuff to me that I neither said nor think, and I don't understand why such negative reactions to questions.
There are great tanks that tweak based on ICP results, there are also great tanks that don't. Like I said earlier, and I think you laughed at me for it, there are a million ways to do a reef and they all can work and I think that has got more to do with paying attention to the reef rather than the particular method.
We all want ICP to be what people say it is, and we are asking questions to help see if that is the case, but when what should be a discussion degrades into a fight, I try to disengage for my own well being.

 

[Reefahholic]

Don’t get me wrong - I want ICP and a results to dosing interpreter to be the answer… I am jus not seeing the precision to make the leap.

Do we really need “perfect” precision to achieve the end goal. Here’s results below:


Water changes are not precise at all, yet many still utilize them. Ten trace elements in one bottle are far from precise. What kind of margin of error are you wanting to stay under, and what is the margin of error for your current test kits and how do you know? Can you show me the testing you’ve done yourself to validate them?

We’ve listed the data below about accuracy and repeatability. What would you like to see that’s not posted below:

Correctness and repeatability of ICP-MS seawater measurements

Hello Everyone! This is a „takeout“ from the reef moonshiners thread, which can be found here: https://www.reef2reef.com/threads/moonshiners-method-feedback.996193/ One point raised in this thread is that quality from seawater labs is often seen with a lot of skepticism. I would like to quote...

www.reef2reef.com

 

[Reefahholic]

I think you are attributing stuff to me that I neither said nor think

Did you not say that?

I followed up with:

 

[rtparty]

@Reefahholic

Here was my last ATI ICP taken in March of ‘23

 

[Reefahholic]

there are a million ways to do a reef and they all can work and I think that has got more to do with paying attention to the reef rather than the particular method.

Ok, so “Reefer 1” and “Reefer 2” set up two reef tanks. Both have the same skill level. Both are 20 year veterans.

Reefer 1 only gets Alk, Cal, and Mag, and hobby grade test kits for those 3 elements.

Reefer 2 gets full access to all elements, and unlimited ICP-MS data.

Question: Which reefer will get better growth speed and color. Remember, both have equal ability and experience.

Reefer 2 will win every single time. Why? Because we limited Reefer 1.

 

[Reefahholic]

@Reefahholic

Here was my last ATI ICP taken in March of ‘23

Not bad. A few things I want to point out, because I really want you guys to see this.

First, this is pretty typical for a OES analysis.

Remember when I mentioned Cobalt has a little better sensitivity, and is closer to the LLOD, and will sometimes show up. You can see it here.

Also notice Vanadium showed up. It’s another element that is right at the border of the LLOD.

Now look at all the elements that tend to be consumed more rapidly and what we refer
to as dailies. Typically Fe, Mn, Co, Cu, Zn, Zi, Se, V. Almost all of them are missing or just low on this OES. Not to say they’re not there, but probably just under 0.2 ug/L or lower. Most guys who aren’t dosing will see them either completely depleted or just very low like around 0.5-0.1 ug/L.

Iodine is low.

Salinity is low which is the foundation of everything else.

I’m actually impressed with your Fluoride level. Usually it’s very low if not dosed. I see it come back around 0.20 mg/L quite often.

When you see these deficiencies, how can anybody reefer not want to correct these. It’s hard for me to understand that.

Keep in mind that when you last dosed trace elements, ICP-MS was not available then. It may be something to consider in the future, because the results we’re seeing now are far better than a few years ago. Not to say that correcting most of the chemistry up until that point wasn’t very beneficial, because it was. We just didn’t have the ability to target all the elements like we do now.

 

[rtparty]

Not bad. A few things I want to point out, because I really want you guys to see this.

First, this is pretty typical for a OES analysis.

Remember when I mentioned Cobalt has a little better sensitivity, and is closer to the LLOD, and will sometimes show up. You can see it here.

Also notice Vanadium showed up. It’s another element that is right at the border of the LLOD.

Now look at all the elements that tend to be consumed more rapidly and what we refer
to as dailies. Typically Fe, Mn, Co, Cu, Zn, Zi, Se, V. Almost all of them are missing or just low on this OES. Not to say they’re not there, but probably just under 0.2 ug/L or lower. Most guys who aren’t dosing will see them either completely depleted or just very low like around 0.5-0.1 ug/L.

Iodine is low.

Salinity is low which is the foundation of everything else.

I’m actually impressed with your Fluoride level. Usually it’s very low if not dosed. I see it come back around 0.20 mg/L quite often.

When you see these deficiencies, how can anybody reefer not want to correct these. It’s hard for me to understand that.

Keep in mind that when you last dosed trace elements, ICP-MS was not available then. It may be something to consider in the future, because the results we’re seeing now are far better than a few years ago. Not to say that correcting most of the chemistry up until that point wasn’t very beneficial, because it was. We just didn’t have the ability to target all the elements like we do now.

Salinity wasn’t low. ATI is terribly inconsistent and inaccurate with salinity. I learned this in my salt testing.

Almost all the dailies you mentioned are dosed by me. I’m not concerned about their exact level because not one of them can be shown to do something. I just make sure that there is some dosed every day. Again, not worried about the exact number or if it’s “depleted”

I have seen no difference by dosing them individually or in an all in one type solution.

Iodine is usually slightly low or slightly elevated whenever I’ve tested. Again, I am really only worried about it being there and not an exact number to hit.

For me to “want to correct these” would mean there is a specific goal to be shooting for. I would need to see some kind of factual evidence to make these changes and not just some pictures of pretty tanks. I can find pretty tanks from every method out there so it’s just a bunch of anecdotal evidence on what “works best”

 

[Reefahholic]

@Reefahholic

Here was my last ATI ICP taken in March of ‘23

I’m not sure if you saw this so I will post it again.

This was a guy that recently switch from ATI-OES to OCEAMO-MS.

As you can see below, the trace metals were there, but too low for the OES sensitivity. It’s interesting to see this kind of data basically back to back. I know this guy did nothing different than what he normally does, and I’ve seen hundreds like this. Not only that, but I literally watch all their values go from very low or depleted and into our target range. It’s exciting. We only dreamed about this kind of data 10 years ago.

Here’s another new guy. 1st analysis is ATI-OES and the last 9 are OCEAMO-MS. His consumption took off and it becomes a chase for a bit, but you can clearly see he’s making some nice progress by #10. Others do it much faster, but he logged his results nicely.

 

[Reefahholic]

Salinity wasn’t low. ATI is terribly inconsistent and inaccurate with salinity. I learned this in my salt testing.

Almost all the dailies you mentioned are dosed by me. I’m not concerned about their exact level because not one of them can be shown to do something. I just make sure that there is some dosed every day. Again, not worried about the exact number or if it’s “depleted”

I have seen no difference by dosing them individually or in an all in one type solution.

Iodine is usually slightly low or slightly elevated whenever I’ve tested. Again, I am really only worried about it being there and not an exact number to hit.

For me to “want to correct these” would mean there is a specific goal to be shooting for. I would need to see some kind of factual evidence to make these changes and not just some pictures of pretty tanks. I can find pretty tanks from every method out there so it’s just a bunch of anecdotal evidence on what “works best”

I agree with most of what you said, but Andre does keep a subscription to get the latest studies, so we know these elements are important as seen below:

Have you checked the salinity with a Tropic Marin Hydrometer. Mine comes out really close with OCEAMO, but I never tested it against ATI. I always kinda felt it was low myself as my Milwaukee and other equipment would have a fresh calibration and read at 1.026 and ATI would come back at like 33-34. Highly recommend that TM hydrometer. That is one well made tool with a excellent case.

 

[Sisterlimonpot]

This thread has taken a weird turn.

A lot of analytical minds have converged to this thread to try and make sense of how someone can conclude that trace element dosing based on icp results is the reason for their corals growing and coloring up.

Isn't it equally possible that all the attention given to your tank because you're vested in its outcome be the reason?

There are fundamental questions that aren't being answered and they seem to center around transparency. And unfortunately, even if you have a direct line to the proprietor of these icp machines, I don't think they're going to provide answers that are going to satisfy those that are asking.

You can't expect anyone to accept photos of a tank as proof.

@Reefahholic, if you're as tenacious in every thread as you are in this one, I can see why reef moonshine is so popular, you don't back down, and it's clear that new hobbyist will cling to the idea without real proof because of your confidence.

Unfortunately, that doesn't work for those that are more seasoned. Especially when you try to blow smoke with before and after photos and refuse to be open to the possibility that what you're experiencing might be the placebo effect.

I'm not saying it is, but you have to be open and objective to the idea but you're so far convinced, that there's no talking sense.

 

[Reefahholic]

Isn't it equally possible that all the attention given to your tank because you're vested in its outcome be the reason?

Not really. I’ve tried about everything. Nothing has come close to dialing in tight chemistry and eliminating pollution.

There are fundamental questions that aren't being answered and they seem to center around transparency. And unfortunately, even if you have a direct line to the proprietor of these icp machines, I don't think they're going to provide answers that are going to satisfy those that are asking.

Which questions would you like answered. I’ve asked a few times and nobody has replied yet.

You can't expect anyone to accept photos of a tank as proof.

I thought photo’s and video would be about as good as it gets.

thread as you are in this one, I can see why reef moonshine is so popular, you don't back down, and it's clear that new hobbyist will cling to the idea without real proof because of your confidence.

I am confident, but I’d say I’m more passionate than anything. I don’t do things with halfhearted effort. I love this hobby. I love the method. I don’t get paid by anybody. Andre is my friend here in Houston, and I met him a long time ago buying frags and filming his tank. The guy has given almost everybody something for free. You can ask anybody that. He’s never looked at this as a business opportunity. He doesn’t take sponsorships. He’s just a reef junkie like me. We just reef, grow acros, and love it. Simple as that.

I'm not saying it is, but you have to be open and objective to the idea but you're so far convinced, that there's no talking sense.

I’m only convinced because I’ve looked at countless ICP analyses. I’ve seen enough now in my own system to be confident that ICP data (from the right labs) is good enough to make dosing decisions from. I agree that not every lab is good. We must remember that we’re in control, not the lab. We don’t have to act on the data. It’s a tool. I do understand we’re all looking for the most accurate data possible, but unfortunately certified labs would only drive up the cost for all of us. The cost would be crazy, and most can’t afford the current pricing.


I’m open-minded, but when it comes to growing corals I have to be completely convinced that what Joe is doing is better than what I’m doing in order to adopt his method. In other words, If how I’m obtaining data is imprecise, there should to be a better methodology offered. You might say, “that’s hobby grade test kits.” I really don’t believe that changing from 50 elements to 8 is a smart move even if the margin of error is 2% and I’m at 5%.

I’ve showed real world results based on ICP data from people in our group. I showed the results in spreadsheets and data comparisons, but also pictures and videos. I have a lot more. This thread was provocatively posted in all caps, (“HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??”) as if the goal was to prove (and point the finger if you will) that ICP isn’t what people think it is. Maybe it’s not, but it’s definitely good enough. Of course there will be some pushback and dialogue from those of us that perform ICP testing on a monthly basis, and rely on that data to run our systems. Our reefs are doing well. To imply that all ICP data is inaccurate or useless is not correct. There were only 8 elements tested. What is the data for all the other elements you didn’t test? If the data was too inaccurate, we’d all have crashed tanks with no color or growth. That’s obviously not the case. I proved that. For me the only thing I’ve heard is that ICP data is slightly less accurate than 8 hobby grade test kits, but I didn’t catch the actual number/percentages.

We can agree to disagree and I’m not just speaking to you only in this reply Sisterlemonpot. I’m speaking to all that we’re engaging and don’t trust ICP accuracy. I’m referring to the accuracy of OCEAMO alone, but I did have decent results with ATI. It’s not fair to put them all in one category and speak about them all as if they are one. Each lab is far different from another. They have different machines, software, environments, plasma interfaces, operators, etc.

Let’s do this. Let’s get it back to the topic at hand. I have an honest question.

What is the margin of error for OCEAMO specifically compared to the hobby grade test kits. The actual percentages. (Example…hobby grade test kit = 2% and OCEAMO = 5% etc.

Also, plz let me know what questions you have as well that aren’t outlined in the other thread. I will do my best to get the answers for you. I am a man of my word. Even if the margin of error turns out to be higher than we would like, I’ll be honest and list it here if I can get that data.

 

[Reefahholic]

I think this is what they’re looking for, but Christoph has already answered these questions. Nothing will ever be enough.

Arsenic: Certified value 1,90 µg/l (uncertainty 0,13 µg/l)

Cobalt: Certified value 0,074 µg/l (uncertainty 0,011 µg/l)

Copper: Certified value 0,87 µg/l (uncertainty 0,13 µg/l)

Molybdenum: Certified value 12 µg/l (uncertainty 0,6 µg/l)

Manganese: Certified value 2,47 µg/l (uncertainty 0,11 µg/l)

Nickel: Certified value 1,04 µg/l (uncertainty 0,16 µg/l)

Lead: Certified value 0,098 µg/l (uncertainty 0,01 µg/l)

Zinc: indicative value 4,6 µg/l (uncertainty 0,6 µg/l)

Selenium: indicative value 0,06-0,094 µg/l (higher uncertainty in CRM)

Uranium: not certified in CRM, still shown to show stability


Thank you Dan! Unfortunately you will most likely not like my answer to your question:

Asking for accuracy and precision for every individual element in each individual run is very easy - but to deliver this data in a meaningful way is unfortunately almost impossible.

The CRM/QK data i provided in my first post give a good indication of accuracy and precision (accuracy: how close are we to the specified concentration, precision: how high is the variability between results).

I could now take this data and say (just as example) for arsenic we have a %RSD of 6,5% (calculated from the standard deveation and mean value of above 18 data points). This 6,5% RSD would however not be valid in general for arsenic. If the concentration of arsenic in the sample would be higher, the %RSD would be lower (better statistics) - if the arsenic concentration would be lower, it would be the other way round. Also factors such as salinity have an impact, if a sample would have very high salinity, %RSD would generally be higher.

To provide precision data for each individual analyses it would be rquired to run every sample several times to allow for statistics. This is however very time consuming, and would thus also increase cost very significantly. It is impossible to give data on accuracy for each individual result, because i do not know the actual ("true") concentration. - Accuracy can only be checked with CRMs or other control samples (data shown in my initial post).

Data that i can offer is the replicate %RSDs: In ICP measurements (OES and MS) we are not measuring a single data point, but 3-5 replicates within a short time span. This replicates also allow to calculate a %RSD and thus give a hint towards precision. However the only error range that arises from this data is machine error (sample uptake, plasma fluctuations,...), and does not show any potential error that is arising from sample preparation or calibration. The CRM data (initial posting) does indicate errors from the whole analysis procedure. If you are interested in specific replicate %RSDs for your oceamo measurement, just let me know - i can look them up. Having this data on every analysis report would imo be overwhelming and not very useful.

I hope now it makes more sense to use things such as CRMs with repeated testing to assure quality in general.

All the best, Christoph

———————————————————

We are using certified reference material ERM CA-403 for internal quality control in our ICP-MS seawater runs. This material is issued by the European commission, and thus fulfills highest quality standards. You can read more about this reference material (including the certificate of analysis) in my post here.

The CRM is provided from an outside source with clearly defined parameters of those trace elements. It is not used for calibration, but for internal quality control.

You can think of it as a sample that we are measuring every time we are turning on the ICP-MS to measure customer samples. The 18 points represent measured values of those elements at consecutive measurement-days from late May up until now (we are not measuring daily). I did not skip a single measurement-day.

The CRM is measured after the calibration (which is done every measurement day - you can only operate ICPs in a freshly calibrated state). The CRM measurement is used to validate the calibration and ensure correct machine operation.

Typical error for macroelements is this in the 2-3% range, if performed well. For us this is still too much - this is why we are measuring Na, K, Mg and Ca in addition with cation chromatography since several months, which is even more robust and brings error down to the 1% range. We are also having a control sample (SST, system suitability test) we are measuring there in every sequence. Will post that data graph when time permits...

 

[Pod_01]

I will say it, I don’t like home testing … I have multiple copies of few home test kits and they all give different results. For example Alk, Hanna is 7.8 / Fauna Marin titration test is 7.1/ Kh Director 6.5…
hmmmm…. The truth is there is Alk in that water, the fact is I just cannot nail down scientifically how much there is….
I must be the worst hobby kit iser there is.

I also don’t like the time it takes to do a test just to be confused later…

Anyways back to this question:

Which questions would you like answered. I’ve asked a few times and nobody has replied yet.

Since I like to make a fool of myself,
1) How are these labs measuring the non ICP values?
Salinity
Kh
NO2
NO3
PO4 Orthophpsthate…..

Do they have a dude or dudet measuring drops, looking at colour etc….
I suspect it is automated, all samples I send have very limited water sample, so I just don’t see how they would do home type titration etc…
I know not ICP related but I do like these complete tests, saves me time and headache…. My opinion only.

Some observations/rant, I read about a lot of discussion on individual element (form can be ignored for now) and how that element relates to a coral. Local and sometimes limited discussion in my opinion. The lines I read are if coral doesn’t use it it is useless/ not required etc…

I really don’t see global discussion on how some elements interact or how the particular element can stabilize or provide other benefits.

For example we have Alk, Calcium and Mg. It is accepted that Alk is used so is Calc but very few times did I read that Mg keeps these two elements from reacting and keeps them stable. There are comments that softies need it Mg etc… but in my limited view the real benefits of Mg is to keep Alk and Calk stable. Maybe I am wrong.

I believe same applies to Iodine, fluoride , zinc and Cu. I have observed if I keep these in check my reef tank looks lot better.
This is not related to maintenance done to the tank, I been doing 10% weekly water changes from the start (5+ years).

 

[taricha]

Not that I think this'll be helpful to everyone, but maybe to some.
I just want to try to clarify why Reefahholic and myself/Rick/Dan/others etc are talking past each other here.
The central point of our study was evaluation of both ICP vendors and trusted chemical test kits vs known certified standards in ranges that both can measure.
We found that if you have a trusted kit and are comfortable using it, it's at least equal to or better than ICP for those things. And there's no reason to throw out your own test results because an ICP report disagrees.
This is not a study on the performance of ICP for the other dozens of measured elements, nor on moonshiners, nor on Oceamo's performance at low ppb elements vs other vendors, nor on trending 1ppb elements and below. And it's certainly not a study on which levels of these traces correlate with coral growth, health, and color.
These are all extremely interesting topics.
And we collected no data in this study that is directly relevant to them!
Indirectly, we can say some things about not assuming better performance for things we don't measure than things we did measure.

If we look at the central point of the study, more data from ICP companies wouldn't actually change anything about the overall takeaway regarding hobby chem test kits. So "what more do you want from ICP vendor X?" isn't a question about this study really - just about how ICP does things in general.

That said, we DO love to look at data, info, and methods from ICP companies and discuss them in the context of evaluating their performance. It's very interesting - just wanted to clarify that it's not central to the point of our study, and is just a fun side-discussion (even if it seems to dominate the thread).