Shaping by Successive Approximations

Shaping by Successive Approximations ➔ Gradually change the criteria for reinforcement ➔ Rat pulling the lever ◆ Reinforce them getting closer and closer to the lever◆ Once the keep getting closer do not reinforce previous criteria ◆ You don’t want to over reinforce because then you won’t get to the actual behavior you are shaping ◆ You don’t want to under reinforce because then it's just extinction ➔ Differential reinforcement (RFT) ◆ Some behavior gets reinforced and others don’t ➔ The key to shaping is that behavior is VARIABLE ➔ Response Class (R-class) ◆ Any variation that has a common environmental effect ◆ Produces a consequence ◆ If we have a response class that is affected by its consequences = Operant ● Increase: Reinforcement● Decrease: Punishment Differential can be based on R-Variation S-variation Specific effect (direct) R-differentiation (increase in specific reinforced responses) S-discrimination (increase in specific response based on the presence of S^D) Spread effect (indirect) R-Induction (increase in responses that are similar to the reinforced R but aren't directly reinforced)Not producing consequences S-generalization (increase in responses in the presence of stimuli that aresimilar to the S^D but not the actual S^D) Not all S-Delta, just similar ➔ Horizontal slit experiment (pg 119) ◆ Divided into 15 segments ◆ Each segment has a sensor ◆ Response that we are interested in is a nose poke ◆ The sensor goes off when they poke nose but we are specifically interested in nose poking on segments 9, 10, 11, and 12 ◆ X-axis: position, y-axis: # of responses

➔ Vertical Slit Experiment (pg 121) ◆ Divided into 15 segments, with a sensor in each segment◆ Responses that we are interested in is a nose poke ◆ The sensor goes off when they poke nose but we are specifically interested in nose poking on segments 13, 14, 15 ◆ Base line graph showed that there was no nose pokes from 11-15◆ They then started reinforcing 6-7 and above ◆ Behavior increasing and now only reinforcing 9 and up ◆ Do it once again to 11 and up ◆ Finally in panel E, only reinforced 13-15◆ There is an increase in the specifically direct response so there is evidence of differentiation Rate ➔ Differential RFT of Low Rate Behavior (DRL) ➔ DRL 10 seconds (or any amount of time that the experimenter decides) ➔ IRT: inter response time (time between responses) > t ◆ When the rat presses the lever, it has to wait more than 10 seconds before pressing it again. ◆ Everytime the rat presses the lever, meaning everytime there is a response, the time starts over. ◆ The rat only gets food, if the time between the pressing of the lever is more than 10 seconds ➔ Differential RFT of High Rate Behavior (DRH) ➔ IRT < t (3 seconds in this case) ➔ DRH 3 seconds ◆ When the rat presses the lever, it has to press it again under 3 seconds◆ Everytime the rat presses the lever, meaning everytime there is a response, the time starts over. ◆ The rat only gets food, if the time between the pressing of the lever is less than 3 seconds ➔ DRL vs DRH ◆ IRT > t / < t

◆◆ A stronger behavior takes longer to extinguish - more resistant to extinction ◆ DRL is stronger, meaning its more resistant to extinction◆ R-chain vs. Chunk (TEU: Temporally extended unit) ● Chain can be broken apart ○ R1: S -> R2: S -> R3 ● Chunk can’t be ◆ Function > form ● Doesn’t matter how its done, as long as it's done ● The rat could press the lever with tail, head, legs, doesn’t matter as long as the lever is pressed S^D (discriminative stimulus): R -> C ➔ 3-term contingency ◆ Consequence is contingent on the Response◆ The relationship between R & C is dependent on the S^D ◆ Discriminative stimulus sets the occasion for the response to produce a consequence. ◆ The phone rings (S^D): You pick up the phone (R->C) ➔ Stimulus Control ➔ Discriminated Operant ➔ Horizontal Slit (Modified) ◆ Response is now a lever press ◆ R -> Lever press ◆ C -> Food ◆ However, this can only happen when lights 9-12 on the horizontal slit are on=S^D ◆ S-Delta: Punishment (PNT) or Extinction (EXT) ● In this case, Lights 1-8 & 13-15 are S-Delta Generalization Gradient ➔ Orange, Y+R, Y+r, Y, Y+b Y+B, Green DRH DRL Cumulative Rs Time

➔ The peak is at the S^D and the responses are slowing decreases on both sides ➔ Presence - absence training ◆ Only reinforcing the yellow light, no reinforcement for others◆ No light: response (peck) -> no consequence (no grain)◆ Yellow light: peck -> grain ➔ Testing ◆ Extinction of the other lights ◆ When its on a yellow light, it sets the OCCASION for responding but doesn’t guarantee responses ➔ Discrimination Training ◆ Post Discrimination (only AFTER discrimination training)◆ A clear presence of an S-Delta◆ Green light (S-Delta): peck -> no consequence ◆ The peak of responding is not S^D but rather one similar to S^D in the OPPOSITE direction of S-Delta which is called peak shift. Spence - Inhibition theory Honig et al. ➔ Experiment 1 ◆ When the variations are not similar to the S-Delta, we do not get peak shift◆ S-Deltas cause peak shifts, however only if the S^D variations do not share a dimension with S-Delta then they do not ◆ S^D circle with a line through it

● Variations - circle with line shifted ◆ S-Delta - circle with no line ➔ Experiment 2 ◆ Now measuring variations of S-D ◆ S^D circle with no line ◆ S-Delta - circle with a line through it ● Variations - circle with line shifted ◆ Attention ➔ What features of the stimulus are controlling your response? ➔ Reynolds - pigeons ◆ Red triangle and green circle◆ Discrimination Training ● Red triangle S^D: peck -> grain● Green circle S-Delta: peck -> nothing ◆ Test ● Plain triangle, plain circle, blob of red, and blob of green ● Would not reinforce any of these ● Some pigeons respond to color and some respond to shape ● But only one stimulus controls responding Fading ➔ Gradually changing the features of the stimulus that control responding ◆ Can fade out or fade in features ◆ b and d example R-Learning: Directional response; Turn left, then 2 rights etc

Place Learning: Relationship between spatial stimuli; Left at the fence, right at the post