Ep. 16: ABA Skill Data: Probe vs Trial by Trial

Today we chat about ABA skill acquisition data: probe versus trial by trial! Join us to hear pros and cons of each type of data collection method, as well as ways to track data for each!

✨Join our Facebook Group for a community of like minded individuals & to valuable grab FREE resources to save you time! ⬇️

The Misfit Behaviorists | Facebook

Join the Facebook group for collaboration and freebies: The Misfit Behaviorists

😍 More, you say? We’re here for you! 

• Apple podcast | The Misfit Behaviorists
• Instagram | @‌themisfitbehaviorists
• YouTube | @‌themisfitbehaviorists

👋 Find us!

• Audra | ABA in School  
• Caitlin | Beltran’s Behavior Basics
• Sami | B.A.S.S.

🖱️ Rate, Review, Like & Subscribe so you don’t miss an episode! Showing this love helps us get out to more educators out there!

Transcript

Caitlin Beltran: It just, it has that little reminder. Like I'm sitting here with another human being, maybe enjoying the time, even though it's work time, um, right. And it's just, it's more than just checking that box.

Welcome to the Misfit Behaviorists Podcast. Join your hosts, Audra Jensen and Caitlin Beltran, here to bring you evidence based strategies with a student centered focus. Let's get started. 

Caitlin Beltran: Hey everybody, welcome back to the Misfit Behaviorists podcast. Today we are talking about skill acquisition data and trial by trial data versus probe data, pros and cons to each, and some quick tips for each type. I'm going to be sharing my screen, so Audra, while I pull that up, do you have a preference for one versus the other right off the bat?

Audra Jensen: Not, not the bad. They serve different purposes. I do like kind of using a kind of a modified probe and trial by trial. So like in all of my mastery criteria, like by sets, not the overall goal, but like my objectives and stuff, I'll always say either 100 percent on the first trial or three of 80.

So, you know, if a student has passed a certain objective or a set and they go

get the first one correct to me that tells they already have the skill set and so that's a great time for them to consider moving on so we're not so stagnant for a student who is mastering something already has that kind of learning in their system so we can move on and then the trial by trial that's really good for seeing the overall view or really honing in on where, like, what targets are not being mastered.

If they have a set of, 10 things they're trying to master, and if you're able to look at the trial by trial data, and you can see that it's this one piece right here that they're not able to master, well, let's move on the rest of them so they don't keep getting bored out of their mind and just focus on that one piece. So they serve different purposes.

Caitlin Beltran: You covered it all. So join us next week. No, let's dive into some deeper dives here. So like Audra mentioned, probe data, we are and if you're watching on YouTube, I'm pulling up just an example data sheet for probe data. Just Facebook group as well. but right, this is you're collecting data point on the first trial of the program.

So you give the instruction or the SD, the student responds. They're correct. You're circling a plus. And if this was like a baseline situation, I might say, okay, they have this skill. I'm moving on to probe another skill. Maybe I'm doing this again in the afternoon. If I'm working towards like a four out of five criteria, type of thing for an IEP goal.

But no doubt about it, it's a time saver because even if you're practicing that skill one more time or ten more times, you're only having to look down, circle that data point one time. And even though that sounds like not much of a big deal, we know how cumbersome data is and we know how difficult it is to manage when everything, when you're working with students. So that's the first thing. Go ahead.

Audra Jensen: I really like this data sheet that you have. When it's very simple, like there's an a. m. probe and then there's a p. m. probe and that's really good because if you have a student come in and they're fresh and ready to go and a. m. is great for them, but you may check in the afternoon. They're like, I am not retaining that skill at the end of the day, too.

So, and that just gives you those 2 points of time at different settings. It could be different staffing. I like that just 2 single shots in the day.

Caitlin Beltran: was thinking that, too, when I first started using this, that a lot of times we're rotating staff. So if we're looking at, okay, they have this skill, but they always seem to do it with this person and not this person and nothing personal, but maybe there's just something so slight about the way that one instructor is running the program.

And so I always like to get at least if even if we're doing probe, maybe two probes a day, if possible. So another point to probe data, as we mentioned, it's great for initial teaching to establish a baseline. So if you have a skill that you're targeting, such as, just receptive ID common objects or something, and you have in mind 20 to 30 objects that this learner has not shown mastery with.

But you start teaching and maybe you're teaching apple to mastery and then car to mastery, you know in those days and weeks Your learner is also perhaps natural get it just getting natural teaching from their environment. Maybe getting Instruction in other places. So by the time you get to target 12 or 15 maybe they've picked it up already on their own and so that quick probe data For that one day you could really save a lot of time like wow, they had it in the morning They had it in the afternoon.

I'm confident with this I'm not gonna give it those three days that I'm normally looking for Mastery because it seems like they actually already know it at this point.

Audra Jensen: Can you explain the column that says, I think it's criteria.

Caitlin Beltran: Oh, yes, so This is for a school setting where it would be an IEP goal and the criteria is listed either as 80 percent accuracy over 3 consecutive sessions, but with probe data, I usually write it as like a plus score over 3 sessions or a plus score over 5 sessions. So most of our goals either have the probe, the plus score or the percentage, and if I'm working primarily with probe data for that skill, I just put the IEP criteria as a plus or if that makes sense, so I don't have to convert to a percentage and then back again. Does that make sense?

Audra Jensen: Yep. Thank you.

Caitlin Beltran: So another point to using probe data is it's definitely beneficial for students who only need a couple of exposures to new skills to absorb new information. And what I mean by this is we all know our learners best. So I know I can think of a learner. I'm picturing him right now. I can teach, certain things to mastery using a certain type of skill instruction, but it's not going to be in a couple of days. It's going to be more like a couple of weeks if it's a goal that I can see in his skill set in the future. whereas I've also had learners where it's seemingly like they don't know it, but I just show them once or twice and they've got it.

So if I'm working with that type of learner who just seems to pick up new information readily, I don't want to waste time with that cumbersome trial by trial data, converting it to a percentage doing that day in and day out when I can use something like this, check it off every day for maybe 2 days, 3 days and just move on to that next target. another thing is that when we're using probe data, the benefit of not collecting as many data points is not just so that our hands don't get tired. It's so that we can really have that more face time with the learner. Right? So if I'm working with my student and I'm taking data, I'm almost constantly looking away, even if it's just for a few seconds to mark that data.

And so we want to be so diligent with that because we've all been there when, our pen runs out of ink or we lose the paper and we're trying to remember it all in our head, it never works out. So we're always trained to take the data as soon as that happens, as quickly as possible. And so you can imagine if I'm taking one or two data points for each scale versus five or 10, I'm just naturally getting that more face time with the learner. And it can allow for a more personal connection and a more, well rounded session.

Audra Jensen: I use the term, teaching outside the data squares. So whether it's this or the trial by trial, so even though we're taking data and kind of focusing on one thing, that it's really important to get outside of that, to have that face to face time that you can really focus, that gives you that pivotal response. It's that back and forth that gets you outside of just that data point that we get focused on.

Caitlin Beltran: Yes, exactly. It just, it has that little reminder. Like I'm sitting here with another human being, maybe enjoying the time, even though it's work time, right. And it's just, it's more than just checking that box. And then finally, you'll see that some research does suggest that probe data can lead to mastery quicker.

Although I think we'd all argue that everyone is different. And so this really is completely a case for individualization with your learner and your skill. And so in addition to the principle data sheets ,this is a closer up version if you're watching on YouTube. If not, I will link it in the Facebook group so you can check it out.

You can put as many IEP goals up to 10 and then have that AM and PM probe. This is a way that I have moved to a digital version of probe data. So I can either have an iPad or a tablet and be checking this off in session or if I'm using a principle data sheet, I can transfer it at the end of the day or week so that I'm just keeping it more of a visual.

I can shred the paper at the end of the week, and I'm keeping a real running record of how that student is progressing with those skills.

Audra Jensen: Can you put anything in here? Like you'd have multiple goals like you had on the printable one. You can also do it on this with just that?

Caitlin Beltran: Yeah, absolutely. This is a Google sheet that I'm also going to link in the Facebook group. So if you're watching, you'll see it's a template with one goal, and then each day there's room for a yes or no data point check. But you could customize this. You could add the AM, PM columns. You could add more goals to the right. So you, I think the possibilities are endless to how you could tailor this towards your learners.

Audra Jensen: That's awesome.

Caitlin Beltran: And then moving to trial by trial data, this is another variation of that printable data sheet where instead of just that AM and PM probe, you're really recording every single time you practice this skill with the student, literally trial by trial, you're recording whether or not practice opportunity was successful or unsuccessful.

And so this is a more continuous data collection method. It records data on every response or teaching opportunity. And as we mentioned, kind of the opposite of probe data, I would say this one's, this method of collecting data is really useful for When you're working with learners who are slower to progress or, and or, need more systematic prompts embedded into their learning trials.

So I do have a lot of learners that I've worked with and work with currently that, I can give them the skill and then I can correct them if they don't get it right. But I found that using that most to least prompt hierarchy is more successful with them, especially if it's an early learner. So instead of just saying, okay, find apple and then correcting them if they're wrong, I'm embedding those prompts. So I'm maybe on the 1st day. I'm saying, find apple with a gesture and then taking that trial by trial data, waiting for them to reach their goal of maybe 80 percent independent 80 percent accuracy and then I'm fading to independent.

So only after they can do that successfully with the gesture for 3 days, then I'm going to move to independence. They find apple, but I'm still going to record 5 to 10 opportunities, so I'm getting like a really fine tuned measurement of what their performance looks like.

Audra Jensen: The other reason I really like trial by trial data is you can put multiple exemplars, multiple stimuli all into one like set or one objective. So you have a learner, maybe just the opposite of what you've described, a learner who. picks up things very quickly. You don't need to do just Apple in his trials.

You could have, you know, five or ten and then you're taking trial by trial because you're going to have them all randomized. It just makes it so much easier to track that data as a group instead of a single one. 

Caitlin Beltran: That's so true. We do this a lot with, like, in our preschool for doing imitation skills or imitates play. So, a lot of the staff will even write if it's minus they'll write oh, this 1 was for, imitating pouring tea. Whereas, like, the plus was for rolling car or something. So you can see it at a glance where they were making those errors. And like I mentioned, this, I would say trial by trial really gives you that more precise analysis because if you're looking at that probe data sheet and you're saying, okay, they got apple in the morning and they didn't get it in the afternoon, that's the only information you have, right? But if you're looking at this data sheet and you're saying, okay, I was testing them on finding apple and they got it at first, then they had an error, then they got it twice, then they had another error.

So overall, They were 60 percent successful, which is pretty good. More than, half, more than chance. But it seems like they started off strong and then made an error, but the data was pretty variable. So you're really looking at not just that end score, that 60%, but how they did within that actual teaching session.

Audra Jensen: Yeah, that would be really useful if you see like a plus, plus, plus, minus, minus, you can see that they've got some momentum, then they're like, I'm bored with this, or whatever it is, you can analyze your student that way.

Caitlin Beltran: A hundred percent and I can think of a learner right now whose performance almost always looks like that. And that's exactly what we're doing. We're responding to that and saying, okay, we're seeing he just gets bored very easily. And he's over it by the 3rd or 4th trial. So not to say he shouldn't have to say that, prove he that he knows it, but we could also be mixing it up a little bit and making it more engaging.

Audra Jensen: Well, you can imagine he's going, I just showed you what it was three times.

Caitlin Beltran: Fine, you don't think it's that, it's this now.

I was just going to say I have had students where this is where it comes into play. you're using your reinforcement wisely because I've had students where by the third one, they're like, okay, I'm going to change my answer because you clearly don't believe me or you think I'm wrong.

Um, and I think with trial by trial data, I would say there's research to suggest higher rates of maintenance. Again, I think this could be skewed depending on how you look at it and definitely is unique to each learner and each skill, but it did make sense to me because you're naturally taking data more often on more trials.

And I think inherently, even if you're telling yourself, I'm going to do probe data, but I'm still going to practice it a bunch of times. There's something to that accountability factor that if I'm telling staff, I really want you to get at least five, if not 10 data points for this skill. You can imagine it's probably more likely that student is practicing that skill more often than with probe data.

And again, some learners need the additional practice and some don't. So, again, you could cherry pick that data, I guess, as always with data to mean different things. and this one, so here's a closer look if you're watching at that trial by trial data sheet, and it gives you that really precise analysis, and then I always put the percentage in the final column, but if you do want to move this into a digital format, you could always put this into a trusty Google sheet, which I love to do, and here's an example that has the date, so But also up to four or five goals and then I'm just popping in that percentage and that way I'm seeing a visual kind of analysis of how that learner is progressing with all of their skills at once.

Audra Jensen: What's really nice about these Google Sheets, graphs is that you can print them, but you can add them to progress reports. You can send them home to parents. these are just really nice. It's a very nice visual way to see are we making progress on a certain goal.

Caitlin Beltran: Yeah, for sure. You are just love to print paper graphs from digital resources, but it is. It's possible.

Audra Jensen: I like to look. 

Caitlin Beltran: We're going digital and back to me. But yes, it's

Audra Jensen: Well, I think a lot about the, it's the parents on the other side, and I don't know about you, but I do not have a lot of parents who will get onto the Google sheets. Even if we share, we've tried so many times to share out digital and they just don't. But if you send it home in somebody's backpack, they will pull it out and look.

And I just, I know how nice it is to see that.

Caitlin Beltran: That's totally a valid point. And so I think in conclusion for me, my gem, my takeaway this week would definitely be to really think about next time you're choosing a data sheet for a particular learner, instead of just whipping out the last one that you used and then putting that learner's name and skills at the top, really putting thought into what's going to work best for this learner and for each skill.

Like I definitely have learners where we have three or four skills at the top where we're doing trial by trial. And then we have three or four skills at the bottom where we're doing probe data because it just lends itself better. I mean, if you're opening a lunchbox, I'm not asking anyone to open their lunchbox 10 times in one session or one day, even maybe twice would be a stretch.

Putting on coats, same thing. But if you're writing your name or matching a letter, I mean, you can easily do that five or 10 times throughout the day. So definitely check out those data sheets in the Facebook group and explore. See if you can customize them for your learners and find them in the group, but also follow us on Instagram at the Misfit Behaviorists podcast and let us know what you think.

Let us know what you're using and why.

Audra Jensen: Excellent. Thanks for joining everybody. We'll see you next week.

Caitlin Beltran: Bye. 

Thanks for listening to the Misfit Behaviorists. And be sure to tune in next week for more tips and tricks. Don't forget to subscribe so you don't miss an episode.