Anonymous Evaluations Team Initiative
Hi All!
I wanted to post each announcement under a category called Announcements, but it would've gotten too messy with everyone replying to each announcement in the same thread, so I will be making them all separately.
Secret Shoppers are a popular quality tool that businesses often use. They go into the store, and then they buy items while evaluating the service. Once upon a time back in 2016, 7cups had an Anonymous Evaluations Team that would evaluate listeners to identify which could use some training (not because they were "bad", but they were new and inexperienced), and which people were here for reasons that don't fit with the site's mission.
We've been wanting to restart it for many years, but due to various reasons, were unable to. Quality Track leadership team and I are proud to say we will be restarting this initiative. We may call it Secret Shoppers instead as more people know what that is, but the name is flexible. It will restart once we get all the pieces in place.
I've created a draft of what I'd like to do. The basic run down this:
1. We will make it part of the Quality Track projects, so those who apply for Quality Mentor/Teen Quality Star and are accepted can sign up to join. They will have to do a certain amount per week for this project to be effective.
1. Team member reaches out to a non-verified listener via personal request and does a pretend chat of medium dificulty based on a topic from their profile. It would last about 20 minutes.
2. They would evaluate based on criteria such as did they respond, empathy, professonalism, giving advice, etc. Those that are here for the wrong reasons would get reported to admin. Those that are just inexperienced, but genuine, would be informed they were evaluated.
3. The listener would be paired with a quality mentor to do skill building.
Here's where I am getting stuck, and I need your help as a community. I am not sure how to best relay the evaluation infomation in a non-punitive way. I feel like a email might be too cold, but I also know receiving feedback that you need to improve can be difficult and hurt. I want there to be a way to balance compassionate feedback with the reality that every listener (including myself) has areas they can improve in to be more effective as a listener. We're all growing and that's okay. It doesn't make us a "bad" listener.
All feedback is appreciated, and if you have other ideas you think might be good for this, please feel free to share!
That being said, please keep in mind three things:
1. I am new. I've been here less than week in an admin role, so I'm still learning. This won't be perfect right off the bat. It's a draft, and it can be tweaked and changed. I am putting it out to the community because I think communication and transparency goes a long way, and I want this to be something that works. More brains means more ideas that I might not have thought of. We'll start the project once we have all the pieces in place.
2. Please try to keep any negative feedback constructive. I have noticed some forum posts receive feedback that looks like "That's a stupid idea", and that is not very helpful feedback. I'm doing my best to learn the ropes and plan ideas that will benefit the community openly and transparently. Tactful constructive criticism is always more valuable. If you post a criticism, please also add a solution that you think might work better.
3. I may not be able to use all the feedback received. It just depends on what fits together. I still appreciate all of it and may be able to incorporate an idea in the future.
Thank you for your time everyone, and i look forward from hearing from you!
Edit: A lot of great ideas have been brought up! A good point that was brought up is what kind of reward could we give for doing a good job. The idea of a Verified Listener badge (if they meet criteria) or leaving a review as a "reward" is intriguing. Please let me know if you think of any ideas for that.
@EvelyneRose
this is an absolutely amazzingg idea. I think it would help a lot of listener to grow and be better in their role.
I loove the idea and i definitely would like to step up to help.
As new listeners many might not know about mentors and other resources in the site and i have met few listener from both my listener and member account who are hesitant to go to the group for help. And later on i think they end up facing certain issues like lack of proper self care and non existent boundaries which are often stressed and reminded through listener support rooms. Moreover I have met listeners who despite the initial training goes against the rules and do things like demand personal information the member dont feel comfortable to give and blocking if they dont, preaching religion etc.
So an anonymous evalution would in my opinion be wonderful!!
About how to give them the feedback, i think it would be better to give them their own space and time to learn from their own mistakes first before anybody else interfere. A mentor connecting them to just casually check up on how listening is going for them, i think might do.
And yes yes, emails are kind of cold and a huge turn off. The chances of new listeners feeling insecured and wondering if they are doing their job properly is very high so a sudden email saying that they have things to improve might make them feel like they are not fit for the role. So i guess 1-on-1 chats with a mentor would be a better way to give them the feedback as well as appreciation for all the areas they did wonderfully.
Last but definitely not the least, Thank youuu Evelyne Rose for bringing this project back.
Interesting idea! Here are some thoughts:
Secret Shoppers—the name seems OK, but at 7 Cups we like to abbreviate things and the abbreviation would be most unfortunate. I suggest Evaluations Team, ET.
1. OK, a certain number of people can sign up to join the team and they have to do a certain amount per week. How does the projected workload compare with the number of new listeners joining? Is it the intention to evaluate all "430,000" listeners? all new listeners? something else?
1 bis. Personal request means the listener has to be online, but some listeners don't go online. Language is going to matter. Topic is sometimes going to matter. Religion or other belief system might matter.
Not sure what "medium difficulty" means. There's not much agreement about what makes a chat "difficult".
Mock chats are really really difficult to do well, because a skilled listener's intuition can allow them to detect that the "member" is lying, in effect. This can make a skilled listener score badly while a robotic beginner ticks the boxes. Training for the team will have to be good to avoid this problem.
2. I worry that this system isn't going to be easy when there's little agreement about what empathy, professionalism, giving advice actually mean. It seems to me that listeners have very different styles, and that this diversity is good. I would not want ET to see themselves as enforcing conformity. I think it would be better to use this scheme to flag obvious breaches of the rules, and not matters of opinion.
3. Skill building would be good, but only if there is a well-defined lack of skill and the listener agrees that they lack this skill. You can't skillbuild someone against their will.
To relay the evaluation information—I imagine a followup chat with the same person who made the evaluation. This makes the evaluator accountable for the opinions they've expressed. I agree that an e-mail is too cold.
How is the success of this initiative going to be measured? Listeners who get evaluated and skillbuilt could be asked to rate the end-to-end experience. But what of the listeners who get evaluated and leave?
Charlie
@EvelyneRose
This sounds like a great idea :) Would it be possible to do it in a way that doesn't tell the listener they're being evaluated? That would give the most accurate picture of how they listen, since the possibility of them acting differently during an evaluation would be eliminated.
@EvelyneRose
I had what I think is an interesting question or thought. For the listener, a chat is a chat is a chat (if they are unaware they are being "shopped"). Number of people helped on their profile page, number of chats as well as positive reviews.
So, I have always learned that a great leader not only has the authority to have discussions and help to mold or change negative behavior, but also the power to reward the positive behaviors that they seek. Say the listener is "shopped" and does a wonderful job. I'm hoping the secret shopper will also take the time to leave a positive review for the listener. This in and of itself can be a powerful tool for teaching. I think the listener is either rewarded for their time that they spent (catch them being good) or they should be pulled aside.
A listeners stats do matter here and the numbers are on our pages and part of us for a reason. It gives administration, people that may not know us a way to quantify our behaviors when they are not around us. Like it or not, this is a tool used to judge our behavior.
It seems as if 7 cups does take a lot into account and does give a "wide berth" with this.......but in my opinion, fair is fair. Many times I have said that sometimes your members can be your very best teachers. Our members have the power to reward us for a job well done or the power to leave a negative review. Perhaps this could also keep away from only specific listeners from being overly shopped, day in and day out because if they have done well, the reward via a positive review will be on record also.
@SparkyGizmo absolutely, if the listener is good a review would be let, or I think even they could get the "verified listener" badge right away as a result of it if they are very high quality. Good point.
Additionnaly a listener would not be tracked twice, there would be record of listeners that was already screened out, we want cover a maximum of people with a team, and not evaluate multiple times the same people.
@TortueDesBois
I love your thought process! 
What a great idea! 
If the listener has not already been verified and does not have the badge then it could be given immediately! Wow! Talk about the power to reward! 
Nice one!
@SparkyGizmo
Yes as Tortue says, there would be a record kept so no one gets shopped twice. I like that idea @tortuedesbois thank you! And maybe not immediately as that depends on human resources available, but maybe weekly or every few days.
@SparkyGizmo The project would focus on listeners that are not verified, i dont see the pertinence to do it on people who are already verified listener, unless you fall on it accidentially when doing through the queue ;)
@EvelyneRose yes by right away i mean at the same time as those are treated. lol
@EvelyneRose I love this idea! I think it is really important for this kind of thing to take place so we can know what is going on in our community. You mentioned that it would be used for new listeners is that exclusive or will it eventually become more widespread? I was also wondering how listeners will be selected for evaluation, will it be based on reviews of the listeners/ member feedback or will the selection process be random?
Either way I think this is an excellent idea, and while yes it is hard to receive feedback sometimes the best way to improve your skills and therefore help more people is via feedback and therefore I think this is an amazing option.
@HappyLittleTr33s
Hi Happy! The criteria all has to be figured out. For fairness, it'd probably be random so you can be any amount of experienced, but it will be all listeners, not just new. I have edited. If only because it's more fair this way so that new ones don't feel they might be evaluated at any moment.
@EvelyneRose
i think this is an amazing idea and would love to get involved as well! A thought that I have about it, is that newer listeners and those not verified should get tested, to make sure, but at the same time a lot of those who get verified act differently during their mock chat than they do during any other chat, so I feel like even verified listeners should still be able to get shopped. I feel like this may go farther In also making sure that those who get verified, are maintaining those skills while with members rather than just when they know someone is grading them.
also for feedback, I do agree that having a mentor reach out to them regarding their feedback would be the best approach, as long as those giving the feedback are careful about using the layering and also making sure to include what they did well.
another thing I've noticed, and what I think would be something to consider is that it's really easy for a lot of people to connect accounts, especially if they either already knew the mentor, or the same mentor reached out to give feedback, so there would still need to be some way to preserve the confidentiality, whether it be someone else giving the feedback, or anything else that would protect the mentors as well.
a question about the process to consider, if it's going to be requesting a chat by personal request, does that also mean that anyone who keeps their status as offline or busy can't be shopped? A big portion of listeners are always marked as offline or busy, including a lot of those that would seem to need to be shopped through this as well.
@GrahZeymahzin
Thank you Grah! Good question on the offline part. Will have to figure that one out.
@EvelyneRose I really like where this is headed.
I see your point about how to deliver the feedback from this. If it just shows up out of the blue, it could cause a strong "We need to talk" anxiety response. I'm wondering if it could show up as a notification. Something like, "You have new feedback from a Secret Shopper" and a link that they can click on or not as they wish. Kind of the feel of the notification of a new review. Coupling the feedback with a positive review seems like a win too. And maybe even a badge. IOW, throw in enough positives that it lessens the anxiety of getting reviewed.
