Creating More Conscientious Interviewers
 
Home
Accolades
Programs
Online Training
Demo Video
Community Forum
Articles
About Jay
Letter from Tim
Order Now!
Evaluation of On-the-Job Behavior Change Results
of Interviewing With Style by Tim Gard
Presented May 20, 1999

Income Maintenance workers who attended the ‘Interviewing with Style’ training were asked to use a Training Impact Evaluation for the course.  To enable data gathering for this Evaluation form, workers were given the assignment of applying any techniques presented in the course in three separate interviews. 

The goal of this evaluation is to assess how likely it is that the Interviewing with Style training results in behavior change. 

On the whole, worker responses indicate that the training had a very positive impact --some behaviors were changed, new behaviors were learned and applied.  The Council Bluffs Region is fortunate to have many years of experience represented in its IM staff.  Some of these experienced workers state in the evaluations that they already knew the techniques presented in the course or noted on the form that applying the technique was not a change in their behavior.  For others, the techniques were very new and comments on the forms were enthusiastic. Because anecdotal evidence helps make reports more real, some of these comments are reproduced at the end of this report.

Learned behaviors most often tried and reported were

¨      Greeting the client with a handshake and introduction—workers observed that greeting the client and engaging in some small talk helped put the client at ease and helped establish rapport.

¨      the power of silence—some workers reported realizing that they had previously been “filling in the blanks” themselves.  This technique helps clients realize the worker is going to wait for an answer that they are responsible for.

¨      open ended questions—this technique was reported as leading to more accurate information

¨      Explaining what is needed and why—workers noted positive differences in client cooperation and in interview progress when the reasons for the interview and its questions, and how these related to client eligibility were explained first.

¨      gaining understanding of the client’s intentions, beliefs, motivations—workers wrote that this helped them understand the client’s responses and actions, and better work with the client

¨      having eye contact, showing an interest—workers reported that the client was more willing and the interviews went better when these were used

¨      investigative techniques –when appropriate, these techniques were found to work, improving accuracy of information (though not always client satisfaction) and

¨      closing techniques—various ways to cue the client that the interview is ending were tried and valued

45 workers returned Evaluation forms for three interviews, 4 workers returned forms for only two interviews and 5 workers returned forms for only one interview.

This means that results were reported for 148 interviews.  There were multiple response areas on the form and workers often listed more than one response for an item on the form.

The Training Impact Evaluation form lists these under ‘Topic Trained’ down the left hand column:  Contact and control (instant rapport, getting to yes, power of silence); Framework (boundaries, frame of reference, truth and consequences); Environment (credibility); Understanding of client (intentions, beliefs, motivation); Detective work (questions, investigation); Counseling, coach; and Closing the Interview.  The other columns are ‘How used in Interview?’; ‘Change from previous interview behavior?’; and ‘Impact:  positive or negative?’ 

The form turns out to have been too complex for use ‘on the job’, well after the training session.  Workers did very well at reporting techniques applied, fairly well at naming the techniques but not as well at reporting those techniques consistently in the seven categories.  If workers have learned and applied interviewing techniques presented in a short training session, whether or not the techniques are understood or reported in a certain category is not important to evaluating results.  The responses then, are reported and subtitled here by technique applied but the report does not divide techniques by the seven categories.

Number with this response

Subtitle/

How used in interview

Was this a change for worker?

Was the impact positive or negative?

 

Introduce self, offer  handshake

 

 

16

Offer handshake, introduce self, make small talk on way to office, etc

Yes

Positive

4

Offer handshake, introduce self, make small talk on way to office, etc

No

Positive

 

Explain what’s needed and why

 

 

25

Explained what and why

Yes

Positive

4

Explained what and why

No or blank

Negative / NA/blank

11

Explained what and why

No

Positive

5

Explained what and why

Blank

Positive

1

Discussed needs

Unknown

Positive

 

Gain an  understanding of client

 

 

6

Gain understanding of client intentions, beliefs, motivation

No

Blank or NA

1

Gain understanding of client intentions, beliefs, motivations

Yes

Negative

44

Gain understanding of client

Intentions, beliefs, motivations

Yes

Positive

7

Gain understanding of client intentions, beliefs, motivations

No

Positive

9

Gain understanding of client intentions beliefs, motivations

Blank

Blank

3

Gain understanding of client intentions beliefs, motivations

No

Negative

1

Gave referral info pertinent to client situation

Blank

Blank

3

Gave referral info pertinent to client situation

Yes

Positive

2

Gain understanding of client intentions beliefs, motivations

Blank

Positive

 

Starting w easy or yes answer questions

 

 

5

“Columbo”-start with easy questions

Yes

Positive

1

Start with the yes answer questions

Yes

Positive

1

Start with the yes answer questions

Blank

Positive

 

Eye contact

 

 

24

Eye contact, show interest in client and what they say

Yes

Positive

1

Eye contact, show interest in client and what they say

Yes

Blank

5

Eye contact, show interest in client and what they say

No

Positive

2

Eye contact, show interest in client and what they say

No

Blank

 

Speech (of worker)

4

Slow and or soft speech, calm tone, smile as appropriate

Yes

Positive

1

Slowed when going over what is for worker repetitious information, checked for client understanding

Yes

Positive

1

Slow and or soft speech, calm tone, smile as appropriate

No

Positive

 

Listening

 

 

1

Mental movie of client situation

Yes

Positive

1

Repeated what client said

Blank

Blank

2

Leaning forward to listen

Yes

Positive

1

Asked question and then sat quietly to let client answer

Yes

Positive

6

Listening intently, not finishing sentences or filling in the blanks for the client

Yes

Positive

 

‘Instant rapport’

 

 

5

‘Instant rapport’

Yes

Positive

5

‘Instant rapport’

No

Positive

1

‘Instant rapport’

No

Negative

 

Power of silence

 

 

11

Power of silence

Yes

Positive

4

Power of silence

No

Positive

 

‘Getting to yes’

 

 

1

‘Getting to yes’

Yes

Positive

1

‘Getting to yes’

No

Positive

 

Focus

3

Letting client ramble, repeating what is said, confronting discrepancies

Yes

Positive

1

Asked Has anything changed since this application was filled out?

Positive

1

Focus on the business at hand

No

Positive

 

Establish control

 

 

6

Established control with boundaries

Yes

Positive

1

Established control with boundaries

Yes

Negative, couldn’t

2

Didn’t get off the subject

Unknown

Positive

1

More control

Unknown

Positive

3

Pacing, leading

Yes

Positive

2

Established control with boundaries

No

Positive

 

Open ended questions

 

 

19

Open ended questions

Yes

Positive

2

Open ended questions

Yes

Blank

1

Open ended questions

Yes

Negative

 

Investigative

 

 

3

Cover, move on, go back and review what was covered earlier

Yes

Positive

1

Tried to make the story complete w more questions

Yes

Negative-

Couldn’t

2

Tried to make the story complete w more questions

Yes

Both positive and negative

4

Tactfully asked in various ways til got answer

Yes

Positive

3

Asked what has happened since last we visited

Yes

Positive

5

Continued to ask  for answers to unanswered questions

Yes

Positive

2

Continued to ask  for answers to unanswered questions

No

Positive

1

Continued to ask  for answers to unanswered questions

Blank

Blank

12

More direct in my questions

Yes

Positive

1

Asked what changes do you foresee?

Yes

Positive

1

Asked follow-up question to see if another change had happened

Yes

Positive

1

Called client’s bluff/verified info

Yes

Positive

re accuracy

 

Frame of reference

 

 

2

Frame of reference

Yes

Positive

1

Frame of reference

No

Positive

 

Truth and Consequences

 

 

1

Truth and consequences

Yes

Unknown

5

Truth and consequences

No

Positive

3

Truth and consequences

No

Unknown

5

Truth and consequences

Yes

Positive

4

Truth and consequences

Blank

Unknown

Attention to client non verbal

Communication

1

Attention to client’s body language

No

Positive

1

Attention to client’s body language

Yes

Positive

 

Environment

 

 

1

Tried to create environment of trust and respect

Yes

Positive

2

Tried to create trustworthy, credible

Environment

Blank

Blank

2

Tried to create more professional environment

Blank

Blank

3

Set a comfortable, relaxed environment

No

Positive

5

Set a comfortable, relaxed environment

Yes

Positive

3

Cleared desk of other work

Yes

Positive

1

Friendly

Blank

Blank

1

Hung a poster in office about reporting

Yes

Positive

 

Closing

11

Asked “anything else you need/want to tell me?”

Yes

Positive

2

Asked “anything else you need/want to tell me?”

Yes

Unknown

1

Summarized

Yes

Blank

3

Gave all written info, stressed reporting

Yes

Positive

1

Used redirecting and focusing

Yes

Positive

1

Emphasized importance of PJ

Yes

Unknown

6

Stood to signal end of interview

Yes

Positive

2

Stood to signal end of interview

No

Positive

2

Stood, talked about next steps

Yes

Positive

1

Stood, talked about next steps

No

Blank

1

Reviewed, talked about next steps

Yes

Negative

2

Reviewed, talked about next steps

Yes

Positive

6

Stressed reporting, contacting us

Yes

Positive

1

Stressed need for communication

Blank

Positive

1

Reviewed options, encouraged to stay in touch with us

Yes

Positive

3

Thanked for coming

Yes

Positive

.

Miscellaneous

 

 

2

Maintain positive attitude

Blank

Positive

2

More prep time for interview

Yes

Positive

7

Encouraged client re positive actions

Yes

Positive

1

Gave advice on our services

Yes

Positive

2

Sympathized with client’s situation

Yes

Positive

1

Gave client an ‘out’/ “take app home and discuss this”

Blank

Positive

3

Workers expressed concerns if DHS management will back workers up esp re investigative interviewing techniques

--

--

3

Workers expressed concerns re time allotted for interviews

--

--

1

Referred client to another agency

No

Positive

 

Response either lacked usable information or stated no changes made

10

No changes made or impact found to date

Blank

Blank

19

Response that could not decipher OR Response reported on client behavior without explaining techniques applied

Na

Na

6

Form response area was all blank or noted ‘none’

Blank or ‘none’

Blank or ‘none’

Sub Totals

All

Was this a change?

the impact?

275

 

Yes

Positive

56

 

No

Positive

5

 

Yes

Negative

2

 

No

Negative

6

 

Yes

Blank

12

 

No

Blank

52

 

Blank or unknown

 

ODS summary

Of 408 responses, 275 indicate that a technique that represents a change in worker interviewing behavior, was applied with a positive impact.  This is a very high proportion. 

56 responses indicate that the technique applied with a positive impact was not a change for the worker.  It is unknown whether workers interpreted this as a change from usual practice OR or a change from things the worker had learned earlier.  Careful reading of each form suggests that it probably is a combination.  Either way, positive impact is reported for the technique.

While labeling of an impact as positive or negative is subjective, independent measurement of interviewing impacts would likely also be subjective.

The instances of negative impacts reported is very low for an evaluation based on three interviews per worker where multiple, often new-to- the- worker, techniques were applied.

Evaluation strategies did not ensure that the various techniques from the training were equally applied.  The choice of techniques to apply was the worker’s and since real clients and real work was involved, that seems appropriate.  It may also be that the composite view of workers’ choices reflected here is also a view of the most generally useful techniques.

Organization Development and Support welcomed this opportunity to evaluate impact.  In the field of Evaluation of training, this is rarely accomplished.  ODS is new at the task but whatever expectations we might made have had about impact, would not have been for it to have been this significant.  We believe this report can be very useful to IM Training Redesign, especially in design of Advanced courses.  This report is also being shared with Tim Gard per his request.   

Finally, we must acknowledge that this report is very, very overdue.  We thank Council Bluffs Region for demonstrating considerable patience and we humbly apologize for the delays. 
 

Some worker comments on positive results

“I have a problem of not guessing what the client is going to say and I have helped them finish a sentence.”

[The] “client threatened to walk out and go home to call Des Moines.  Proceeded to explain to him that he would be required to attend another interview

……..so to save his time, we could continue.”

“I like these changes but have to reprogram myself to change the routine.”

“I have really changed the way I interview since this training.”

[The client] “revealed resources not on the application”

[I] “was able to grasp an understanding of what client needed .  Really.”

“I feel this class should be given to all new employees before they start interviewing.  Some times it’s hard to change our ways after 20 years. “

“I have changed the way I sit-instead of completely talking and writing, I now look directly at them and listen. [Have] more eye contact.  Let them talk and not talk for them.”

[before] “usually rambled fast when explaining the program requirements as it was so repetitious to me”

[while waiting, client was mad/upset and ] “had upset the clerical a lot with their comments.  I asked if there was a problem.”  {There was an unrelated problem and] “I  listened to her briefly, then explained I would probably be asking her some more questions that might upset her. Gave her option of rescheduling or going on……She even apologized to clerical when leaving.”

“Felt the [training] day was very well spent”

[Before]” I only assumed I knew [the client’s] intentions were kept within program of application.”  [Now] “I can/do understand their needs better by my opening to the interview.”

   
Call Today - 1-800-865-9939
info@interviewingwithstyle.com