Testing Ammo

[linekin]

I appreciate everyone's thoughts on testing. I'm just looking to see if I may be able to find a better way.
When I said I like a little breeze, I meant just that. Just some, maybe 5mph or less. I just seem to struggle in calms sometimes. I don't mean through the whole target, but just some unexplained droppers. Never far off just out. I've been baffled by calms, indoors & out.
Landy, I've always been interested in the On Target software. I searched a bit last night on some posts on RA about it, I guess I would need it, & a good scanner in order to use it. Then an understanding on how to use both. Not at all opposed to using it, just figured I could find what I'm looking for without it. Maybe not.
I also agree with you about buying test bricks if it were feasible, as I have had countless times when my favorite ammo in whatever gun preformed a skoosh differently when I opened the next box. Not unexpected as I question any rimfire ammo's consistency period. Maybe the software would be a help over that large a test.
Again, thanks for everyone's .02c

Keith

by the way, in my testing of the recent lots I had I thought one lot held promise. When I went back the next evening under the same basic conditions I felt differently.
That's what I'm trying to avoid buying. The search continues.....

 

[Az_Speed]

Hi Landy,

Very interested in your comments as follows:

I'll outline a plan where you can satisfy that curiosity and also improve the odds of successfully testing for lots or quantifying the precision of your rifle.[/U]

Couple of questions...

Is this your plan?

In my case I'll use every single rd in a box of ammo and shoot either 2 25-shot targets or a single 50-shot target. I'll foul the bore with odd lots of the same ammo brand so not a single round is wasted and I'll base my decisions on the statistics best suited for analyses of shot distributions.

If anyone chooses to use "OnTarget TDS" you'd primarily look at ATC (Average To Center), also called MR (Mean Radius).
If anyone chooses to continue shooting groups in lieu of software, that's fine also. Using ATC on a shot distribution has a higher correlation to precision that does shooting a series of 5-shot groups and calculating the average size, but the difference isn't huge and remains a viable option. Even if shooting groups, I'd still get the free version of "OnTarget" for measuring those groups and I'd ask or learn how to properly measure groups because it's pretty damned obvious most don't know how to do so.

Also, as you are testing are you aiming dead center on all 25 bulls as you work the target or are you holding for conditions?

Lastly, can you give an example of “statistics best suited for analyses of shot distributions”?

Thank you,

Steve

 

[HuskerP7M8]

I'm swamped with harvest and have some personal commitments thru the first part of next week.
I'll try and make a few comments after that for everyone.

Landy

 

[Az_Speed]

Thank you. I look forward to it.

 

[HuskerP7M8]

Keith,

In your first few posts you explained how you lot test and to a lesser extent how you evaluate a rifle's performance. You then asked how others do the same and if you were leaving anything on the table that might improve your chances for success.

I'm definitely not going to twist anybody's arm to force them to do anything differently or spend a lot of time trying to convince anyone that the way I do a lot of this stuff is the "only" way. Quite frankly what you're doing is pretty common and many shooters have been highly successful doing approximately the same.

I thought what I'd try to do was dissect a few of the comments you've made in this thread and hopefully leave you with a few more options you might want to consider.
Unfortunately, I'm also going to bore you and others attempting to explain why some methods of testing are better than others and the reasoning behind it.

Let's get started with the thoughts you had concerning the conditions you like to test under.
You prefer a light consistent wind rather than no wind and I don't have a huge problem with that, but I prefer no wind using an indoor facility that has consistent conditions where different and varying degrees of mirage are minimized and easily managed. Successful shooting at indoor tournaments is driven by the ability to recognize mirage and manage it.
Your comment about Piney Hill is much the same as many say about the Barn and I experienced that first hand at the ARA Indoor Nationals a couple of winters ago. The last target I shot was jumping around like a Mexican jumping bean due to the worst mirage I think I've ever seen. If I had to test under those conditions, I'd choose 20 mph switching winds rather than deal with it.
Those conditions are pretty rare though and the majority of the time testing in those locations leads to excellent results. I know because I've shot or tested in nearly all of the indoor matches at least once and in many cases multiple times. Plus, I've shot either in the middle of the night or in my own dedicated testing facility I built over the last 35 years.

To be complete, let me also tell you why I believe some think shooting indoors results in less than expected scores under what should be ideal conditions. This also relates to the following comment by Az_Speed which isn't quite correct:

"I'm not quite sure how ammo can shoot great groups and then not score well. It doesn't seem to make much sense, can you explain?
How does a lot of ammo that consistently shoots where the rifle is pointed not score better than a lot that does not shoot as well (group)?"
The part that's not correct is in red above. It's not correct because the rifle never shoots where it's pointed when shooting groups. By that I mean as you shoot itty-bitty groups that would all fit in the ARA 100 ring or touch the X on the IR50 target, they don't form at the same location from group to group. The mathematical center of each of those itty-bitty groups impacts in a different location and fools you into believing all 25 shots on target should land inside the 100-ring or touch a dot.
It's obvious to see with a poor shooting rifle, but it's very difficult to see without a tool like "OnTarget" when shooting a rifle capable of 5-shot groups in the high 0.1's to 0.2's.

Because I know there will be questions or confusion over what I've said, I'm going to present a screenshot from one of my customized spreadsheets that I hope clarifies.
This image was originally published in the "Fouling Shot" when the President of the Cast Bullet Association asked me if I'd allow him to use it in their magazine. I've published dozens of articles over the years but don't bother searching for them because most of them are under fictitious names or credited to anonymous authors to honor my requests.

It should be self explanatory, but I'll add a few comments that may or may not be helpful.
I used the TDS #7 target which is scaled down to fit on standard sized paper, but the scoring ring diameters are exactly the same as in the regulation sized target.
The 5-shot groups are color coded and composed of the first 5 shots fired on target followed by the second 5 shots, etc.
The "x center" and "y center" are the mathematical centers of each 5-shot group and the average size of the 5x5-shot groups is 0.169". The composite group of all 25 shots is 0.219".
The location and velocity for every shot is also listed.
The actual target I shot is shown below the graphic.

This is my current tunnel record so I don't think anyone can argue I didn't use a rifle that's good enough to get valid results. IR50 250 25X, ARA 2500, PSL 2500 18X.
In any case, this is what all shot distributions look like whether their shot at multiple POA's (Competition), held in a fixture at an ammunition testing center with a single POA, or in one of my rail-guns at a single POA.

I've barely scratched the surface of what I wanted to say but I've run out of time and it's quite possible I've said more than anyone wants to listen to anyway.
I'll try to continue when possible, but it sure as Hell won't break my heart if I'm told to cease. LOL

Landy

Composite Excel Analysis:

Target:

TDS:

This is just for fun. It's an animated GIF I created that I find mesmerizing. It shows the random order of where each lands in the sequence they're shot using two ring diameters sized to show where the shot is an x or 10 on the IR50 target.

GIF:

 

[tim]

Landy, always well thought out and informative.
Give me your thinking on one item I have often wondered about, never gotten any understandable answer to since you mentioned indoor matches.
What are your thoughts, if any, about the fact that some accomplished shooters seem to utilize indoor and outdoor ammo ?
Since I only shoot outdoors I have only 2nd hand information in that regard.

As a follow up and the reason I posted the question in #2, it seems like some agreement, you can test a gun, you can test ammo, but it is tough to do both simultaneously.
It seems to boil down to the concept that you should ammo test with, ideally, a proven rifle and a rifle with verified ammo, hopefully a couple lots.

 

[linekin]

Thanks for taking the time Landy. I know you're busy this time of year.
You'll never ever bore me though with any of the data you post. I've always enjoyed reading, not necessarily understanding it, your posts. Please add, change, derail or whatever any thread of mine you wish! I'm always fascinated!
I can see using the software where group shooting is measured & has actual math put to it. There is no doubt once used you'll know the best shooting ammo you've tested. The #'s don't lie & you can put that into real world target scores.
But with me, without the software the way I do it seems like the only way to achieve similar result over a larger # of shots. In my case a few groups, then a whole card. Then again just to be sure.
Again, I appreciate you taking the time.

Keith

 

[JerrySharrett]

Great info guys. Question-what is a suitable barrel cleaning method to use between tests?

.

 

[Az_Speed]

Great post. Thank you very much. I need to spend some time studying this.

Steve

 

[tim]

Great info guys. Question-what is a suitable barrel cleaning method to use between tests?

.

Since this has been hanging for a couple days.
Two options....wet patch, couple drys, light oil or not, dry patch.
Second option, brush with solvent, patch out, light oil or not, dry patch.
Every few hundered check/clean throat of carbon/lead.

Either way, a well maintained decent barrel should settle right in with 5-6 shots.

 

[HuskerP7M8]

I've always enjoyed reading, not necessarily understanding it, your posts.

You ain't the Lone Ranger! Judging by the number of PM's, emails, and the chatter within the threads when I participate....there are times I feel like no one understands what the Hell I'm trying to say! LOL
In all fairness it's complex subject matter that most are totally unfamiliar with and I struggle with it much more often than many of you probably think I do.

Please add, change, derail or whatever any thread of mine you wish! I'm always fascinated!

Wow! Free reign to make your life miserable every time you post! LOL

I can see using the software where group shooting is measured & has actual math put to it. There is no doubt once used you'll know the best shooting ammo you've tested.

I wish it was true that I always pick the best ammo when I lot test, but unfortunately that's not true unless there's a rather large difference between my choice and the other ammo I've tested. Even with the methods I employ, it boils down to making decisions based on statistical confidence levels.
I'm 100% confident I'll be correct more often than you, but that's in no way the same as saying I'll be correct 100% of the time.

In the majority of cases, we have to get a little lucky in order to find those lots that might be considered "Killer", because I think everyone is in agreement that it's not possible to lot test bricks vs 1 or 2 bx's. It's easy to find good ammo and reject bad ammo, but incredibly difficult to determine if it's "Killer" and not just "good" before you purchase it.

The #'s don't lie & you can put that into real world target scores.

Real world target scores? That reminds me of some work I've done in the last month or two and I think you and others may find it interesting. It should also assist everyone in setting benchmarks based on groups shot during your testing if your preference is to continue shooting groups rather than buying software and working with statistics you're not familiar with.

I've always been interested in how the statistics I generate during testing translates to real world target scores. What I did was write algorithms based on the data I've gathered that most closely match the scores I've shot in my tunnel, the scores I've shot in indoor matches with little or no mirage, and the outdoor matches I've shot under near perfect conditions.
I could have written the algorithms without using any of my data because the means to do so are pretty well known. The only thing you really need is a specific circle diameter that the shot distribution lands in, or in this particular case the scoring ring diameters of the ARA target to predict potential scores based on which statistic you choose to use for that estimation.
I've also written the algorithms to do the same for the IR50 target, but I don't have the time to do so. Just thought I'd mention that because I believe you shoot mostly IR50 and might ask if I'm able to do that.

A brief, and I hope, simplified explanation of the following charts is probably in order. It may not be simple enough for everyone, but I've already said and will say some things that may horrify a real statistician. I'm sure they'll forgive me considering I'm just some hick in Nebraska pretending to be an expurt. LOL

All 3 charts use the same data from a single barrel utilizing the same 3,375 shots. I didn't look, but there's probably some 30 to 40 different lots of ammo in the data.
The reason I mention the number of lots is because I believe that may be the best way to make a determination of whether or not you should give up on a barrel and/or build and start anew. If after I've tested some 20+ lots and the charts I'm presenting suggest none of the results are competitive, I'll rebarrel or start a new build. If I sell the rifle I've given up on, I'll either rebarrel it before I sell it or tell the new owner they might be better off rebarreling it because the odds are they won't be happy with it. I don't want to lay awake at night thinking I'm guilty of selling junk even if I take a hit financially.

All 3 charts show the relationship or statistical correlation to real world scores (ARA Score/C'd) for testing results using 2-shot groups, 5-shot groups, and ATC (Average To Center) also known as MR (Mean Radius).

The red line thru the data points is a trend line and in this case it's a linear trend line. Excel calculates the best fit thru the data points and creates the line.

R2 is a calculation Excel uses to tell us how well the data fits the trend line. Higher numbers are better.

Correl is the term used in the Excel formula bar and is short for correlation. Whether minus or positive, the higher the number the closer the relationship or the higher the correlation is to predicted scores.
You should notice there is some scatter that prevents you from drawing an intersecting line from the group size average in the horizontal axis of the charts and predict an exact score. This scatter is perfectly normal and can only be lessened with the addition of more data. Considering this data is composed of 135 separate tests and 3,375 total rds, it should be obvious why scores vary so much from target to target even under benign conditions with the same rifle and ammo.

The other important result to notice is how the prediction for real world scores suffers if for some reason you were to decide to use 2-shot groups in your testing. The same happens if you use anything other than groups with 5 to 6 shots in them. Testing using 2-shot, 3-shot, 10-shot groups, etc don't work well when testing ammo or evaluating rifle performance. Again, this is well known but probably unfamiliar to most of you.

There's a lot to digest here and I'm certain there will be disagreement. All I can do is present the data and let everyone draw their own conclusions.

I'd also like some input as to how understandable this is. I don't care if anyone disagrees with what I've said....just want to know how many of you can make sense of it. I'd really appreciate the input.

Thanks,
Landy

2-Shot Correlation:

5-Shot Correlation:

Average To Center Correlation:

 

[HuskerP7M8]

Landy, always well thought out and informative.
Give me your thinking on one item I have often wondered about, never gotten any understandable answer to since you mentioned indoor matches.
What are your thoughts, if any, about the fact that some accomplished shooters seem to utilize indoor and outdoor ammo ?
Since I only shoot outdoors I have only 2nd hand information in that regard.

As a follow up and the reason I posted the question in #2, it seems like some agreement, you can test a gun, you can test ammo, but it is tough to do both simultaneously.
It seems to boil down to the concept that you should ammo test with, ideally, a proven rifle and a rifle with verified ammo, hopefully a couple lots.

Tim,

I don't have a good answer for you.
I'm aware that some shooters bring several lots of ammo to a match even when they're shooting outdoors, or they believe they have indoor and outdoor ammo that may behave differently.
I pride myself in having an open mind, so I won't discount their beliefs and can only relate I haven't experienced the same. When I've found great to killer ammo, as verified after purchase, it seems to shoot the same regardless of the environment. Or, to be more precise, it shoots within the variation I expect.

As regards your follow up question, I think you can do both even if not ideal and maybe something I've said in my post above this is a partial answer?

Landy

 

[drknite]

Thanks Mr Landy for all your hard work. Must say I am envious of your tunnel. The information seems to make sense to me. While I am a relatively new shooter, I do believe that 5 shot group's are more meaningful than 2 or 3 shot groups In trying to sort out ammo. Keep up the good work.

 

[tim]

Tim,

I don't have a good answer for you.
I'm aware that some shooters bring several lots of ammo to a match even when they're shooting outdoors, or they believe they have indoor and outdoor ammo that may behave differently.
I pride myself in having an open mind, so I won't discount their beliefs and can only relate I haven't experienced the same. When I've found great to killer ammo, as verified after purchase, it seems to shoot the same regardless of the environment. Or, to be more precise, it shoots within the variation I expect.

As regards your follow up question, I think you can do both even if not ideal and maybe something I've said in my post above this is a partial answer?

Landy

Thanks Landy. While I suspected as much, being only an outdoor guy, suspicions will have to do.
I still have difficulty with the both approach. You are lucky with your test facility, it is likely far easier to establish rifle and ammo paramaters with a tunnel and software.
Most of us mere mortals that have to establish those paramaters outside in the elements, it seems to have always been more productive to ammo test with a proven( hopefully) platform, and gun test with proven ammo that has a track record in more than one gun. One step at a time.

 

[SigP224]

Landy,

Veeeeeery interesting! Your correlation charts have given me some ideas on how to go about analyzing my own targets for lot testing. I think I understand what you have done but I'm not exactly sure how you chose the shots.

You used 3,375 total shots and your ATC chart show 135 tests. OK, that seems to be 135 ARA cards with 25 shots/card equals 3,375 total rounds. So the data source for your ATC correlation chart appears straightforward.

The 2-shot and 5-shot groups for their respective charts have me baffled, though. How did you choose the 2-shot and 5-shot groups on ARA cards? 5 shots would logically, to me anyway, be using either each row or each column to produce 5-shot groups. But 2-shot groups? Shots 1-2, 3-4, 5-6, etc? What happens to shot 25?

Or do I have everything all wrong??

Testing using 2-shot, 3-shot, 10-shot groups, etc don't work well when testing ammo or evaluating rifle performance.

I was wondering why 25-shot groups, such as an entire ARA card, wouldn't be a more accurate predictor than 5-shot groups, especially since your ATC graph showed such a high correlation. Then it hit me: ATC is more valid than group size. For example, you can have a .100 group (number of shots doesn't matter) but if the group is .500 off center then the ATC of that group is .500 and it would not score well even though it is a tiny group.

Thank you for an excellent presentation of a better way to lot test.

Tom

 

[hawkeye wizard]

Landy,
The problem with posts that are very easy to understand is that they're seldom technically correct.
Thank you for making such technically difficult concepts reasonably understandable.

SigP224,
As Landy noted in his post, "Average To Center" (ATC) is more commonly known as "Mean Radius". ATC is not the distance of the shots from the aiming point. Rather, it is the average distance of each shot from the center of the group.
ATC is a better predictor of score because it includes all 5 rounds fired in the calculation rather than just the furthest apart two rounds as does Extreme Spread.
The advantage of "Extreme Spread" is that it's very easy to measure. Not so with "Average To Center".

Landy,
Feel free to correct my above explanation or to rephrase it in your own words. I'm pretty sure your words would be more understandable than mine.

Do you know the worst things about your posts? There are simply not enough of them. And, they get lost in the masses of other posts.
I'd suggest to the forum administrators that all of your posts be made into "stickies" so we wouldn't have to hunt them down.

Landy,
I'm not even certain how to ask this question. And I suspect that the answer would be merely a judgement call rather than a statistically valid one. But, let's try.
Let's say you had equipment that you felt would be competitive at the national level (would give you a reasonable chance of being in the top 10 at the ARA Nationals).
How many 5-shot groups (measured using ATC) would you need to shoot and what would the ATC need to average to make you comfortable enough to buy a few cases and shoot it at national competitions?
How's that for a totally unscientific question????????? Does the question even make sense?

Hawkeye Wizard

 

[SigP224]

Hawkeye Wizard
Thanks. I overlooked Landy's explanation that ATC = Mean Radius. But that brings back up the question I thought I had answered in my incorrect "explanation" of why ATC is better than group size.

Landy
Why is ATC better than the Group Offset measurement in TDS? If I understand Group Offset, which I may not, it is measuring the distance of the group from center which is the desired point of impact. With ATC being the mean radius *of the group* then your group conceivably could be very small but significantly away from the center of the target and thus score poorly.

Well, to answer my own question, group offset could have a similar flaw. The group could be very well centered on the target but be a very large group and not score well.

It seems to me a combination of ATC and group offset would be most valid. Have you looked at anything like that?

Tom

 

[KimZ]

Linear regressions

Is the third regression (average distance to center vs score) measuring mean POI to center of bull for 25 bulls, then regressing this against that target's score? If so the high correlation makes perfect sense: The smaller the average distance from POI to bull center, the higher the score (score is inversely correlated with mean POI-center distance).

The other two regressions say 2 or 5 shot groups. Are these groups "virtual"? ie sampled from scans of scored targets? If so how are the groups sampled from the targets? Consecutive shots?

Thanks

Kim

 

[HuskerP7M8]

Landy,

Veeeeeery interesting! Your correlation charts have given me some ideas on how to go about analyzing my own targets for lot testing. I think I understand what you have done but I'm not exactly sure how you chose the shots.

You used 3,375 total shots and your ATC chart show 135 tests. OK, that seems to be 135 ARA cards with 25 shots/card equals 3,375 total rounds. So the data source for your ATC correlation chart appears straightforward.

The 2-shot and 5-shot groups for their respective charts have me baffled, though. How did you choose the 2-shot and 5-shot groups on ARA cards? 5 shots would logically, to me anyway, be using either each row or each column to produce 5-shot groups. But 2-shot groups? Shots 1-2, 3-4, 5-6, etc? What happens to shot 25?

Or do I have everything all wrong??

2-shot groups:
My spreadsheets throw out the 25th shot when I input the point data from the TDS #7 target. When I shoot a 50 or 100 shot target, all the 2-shot groups are included.
A good observation on your part to come up with the question.

5-shot groups:
I use the same algorithm as Jeff does in "OnTarget" with my spreadsheets to capture the 5-shot group data. In case you're not aware, you can retrieve the same in TDS by hitting the hot-key "Q" to bring up the composite group display and selecting "Print Data".
I had a PM with a similar question and if I don't forget I'll ask Jeff if there's a "print to file" option or if he can incorporate a pop-up showing it when he writes a new version so you don't have the expense of printing this info. I've never thought much about it because my spreadsheets handle it.

In the grand scheme of things the 2-shot and 5-shot group info isn't all that important since you're using TDS and can utilize ATC/MR.

I was wondering why 25-shot groups, such as an entire ARA card, wouldn't be a more accurate predictor than 5-shot groups, especially since your ATC graph showed such a high correlation. Then it hit me: ATC is more valid than group size. For example, you can have a .100 group (number of shots doesn't matter) but if the group is .500 off center then the ATC of that group is .500 and it would not score well even though it is a tiny group.??

Hawkeye down below answered the slight confusion you expressed and I suspect he's just being kind stating I could do it better. Considering the number of quality posts he's made over the years on forums and most notably in "Precision Shooting" magazine before it folded, he'd be more qualified to edit and improve my use of the King's English than the other way around.

Thank you for an excellent presentation of a better way to lot test.

Tom

Thanks, but it's only better if an individual thinks it's "practical" for their circumstances.

Landy

 

[HuskerP7M8]

Landy,
The problem with posts that are very easy to understand is that they're seldom technically correct.
Thank you for making such technically difficult concepts reasonably understandable.

You've got that right! Easy to understand and technically correct mix like oil and water. If you're trying to convey info in a way that might be helpful to the majority, you walk a tightrope trying to maintain the proper balance. I can't tell you how many times I've landed on my head and suffered injury.

I'd also like to offer some perspective to those of you who might mistakenly believe I'm a brilliant guy with an exceptionally high IQ. When I visit with the truly brilliant.....I feel like an absolute and utter moron. I'm not sure how many of you realize just how far the spread is from what you read on internet forums from the hobbyists like me compared to those who live their lives studying pure science. The majority of you probably struggle with topics like this just as I do, but those who truly understand and pursue careers in this field might characterize my comments as child-like. It's all relative.

SigP224,
As Landy noted in his post, "Average To Center" (ATC) is more commonly known as "Mean Radius". ATC is not the distance of the shots from the aiming point. Rather, it is the average distance of each shot from the center of the group.
ATC is a better predictor of score because it includes all 5 rounds fired in the calculation rather than just the furthest apart two rounds as does Extreme Spread.
The advantage of "Extreme Spread" is that it's very easy to measure. Not so with "Average To Center".

Landy,
Feel free to correct my above explanation or to rephrase it in your own words. I'm pretty sure your words would be more understandable than mine..

As I've already stated, I don't know how I could say it better than you to make it more understandable. The best I can do is add to the above by saying there are two reasons why shooting 5-shot groups isn't quite as reliable as using more robust statistics.
The first is of course what you mentioned concerning using only two points of data and losing the valuable information contained in how ever many other shots are part of the same group.
The second reason is what I touched on previously in this thread. You lose the exact location of the group.

The major reason I test by shooting at separate bulls, just as you would at a score match, is because that's the only way I can capture the point data that obviously contains the location of every shot.
I would much rather shoot thru an electronic targeting system and eliminate paper targets all together, as well as having to move from bull to bull. But when I contacted Meyton many years ago, one of the premier electronic targeting systems, and requested estimates based on the options I needed, I realized I had to put some financial limits on the resources I needed to continue this obsession I have. The costs were more than I could justify for a hobby and I decided to trade my time lost gathering data for dollars.

I really really hate to do this, but considering the vast majority believe that BR shooting for group and BR shooting for score are totally different whether it's CF or RF....I have to comment. I've literally tested this to death and I've yet to see any difference whatsoever between the two.
I alluded to this in post #25 and offered an explanation, but I'm under no illusions that many or most of you will agree with me.

I'm gratified by the comments and kind words, but I've got to opt out for a spell again before I can even answer all the current questions. I'm working on 4 hrs of sleep over the last day and a half and hoping to get finished with harvesting corn tomorrow, but then I undergo two days of laser surgery at a specialist Dentist's office and I'd be lying if I told you I'm looking forward to it!
I like to finish what I start, but it may be a while.

Thanks in advance,
Landy