NUEbies

It's hard to know whether this story is just garbled because of the way Tateru Nino has reported it, or because the Lindens have muddled things again -- or both. I find myself asking: Can the Lindens do anything right? That's going to seem very harsh and mean, but frankly, I'm forced to ask the question. We all have a stake in newbie retention, and we need at least the Lab, that goofy software company posing as a scientific institution, to find the facts, even if they won't tell us what they really are.

You know how they have these things called "double-bind studies" and the use of blinds and placebos and so on in scientific studies? So if you are going to do an "A/B" study, i.e. comparing A to, uh, B, you need to have what we might call "a surgically-clean experimental space". In order not to contaminate the batch.

Here I'll interrupt to tell you my first very stark moment of discovering this principle at my father's lab at a factory in upstate New York, where he explained that these weights and measures that had to measure very fine amounts of materials could not be touched by the human hand, and had to be handled with tweezers, because even a fingerprint weighed something. My little brother and I, goggle-eyed, staring at the brass weights, finding the temptation to handle them overwhelming, but we could hardly be allowed to screw up the entire batch. This concept of the experimenter affecting the experiment is of course called the Heisenberg Uncertainty principle, and Niels Bohr also wrote about it. So what does that mean under our Laboratory conditions?

If you are going to test two venues for how they retain newbies, and you want to see if the Mentors versus Greeters doing specialized tours make a difference, then...you can't speak of "Mentors/Greeters" on the one set of islands, and still have the Mentors coming to the island sporadically, and then speak of Greeters on another set of islands. You really have to be clear on what distinguishes Mentors from Greeters, or Mentors doing this special Greeting tour. That means not having them even do their routine or haphazard rounds, which we see them doing now and then on all infohubs and welcome areas.

Furthermore -- perhaps this is just silly me, a non-scientist -- you can't ask the subjects of the experiment themselves if they think they are now retaining better in SL, or are happier -- you have to really check up on whether they retain by monitoring their log-ons, hours, and most importantly, coming back at 30- and 60-day intervals to see if they are "still there". While some of this material seems to have been gathered, it isn't clear whether anything but the newbies own self-reporting is making this determination that...as Jacek Antonelli even had to point out to Tateru...involved a whopping...3 percent difference (which could be a statistical anomaly or caused by not handling the setting and the data properly). I mean, what is the difference, really, between Mentors and Greeters? And have any of these A/B tests involved "nothing at all" versus "something with Mentors"? Surely they must have done that?

Look folks, why is this so hard? Even I, at my own very lowly and unscientific level perform this simple task:

I watch the people who log on by seeing the last log-ins on the group. Although the group log-on information is imperfect -- it can show when a person last needed to use that group, not when they actually logged on to SL -- I find that most people do need to use the rentals group when they log on because they have to put out an object or open the door that is group-set and probably have to do that at least once a week. So I look at the group and see who "retains". If, after 30 days, somebody hasn't used the group, and they aren't visibly paying as a tenant or known roomie, they are ejected, in part for their own good, so they aren't paying group liabilities, which are part of rent in my system. So each month, I expel anywhere from 12-20 people, some of whom sometimes make their way back after 90 or 120 days (which is why studies of Second Life have to be VERY LONG TERM!) because of COMPUTER PROBLEMS mainly caused by the forced introduction of WINDLIGHT. Hello! Then, if they were tenants, I write them -- did you have any problems? I saw you refunded or didn't pay and didn't log on, etc. I can also eyeball these accounts to see what happened to them -- perhaps they moved to another rentals or got their own land, I can tell from their groups. Or they have some message on their profile, "Good-bye, cruel virtual world...etc." Remember, "Gone with the Windlight..."? That sort of thing.

Unfortunately, these retention studies the Lab keeps doing never seem to go past 30 days, which is hugely annoying. That causes them to make pre-emptive decisions based on incomplete data. (Although getting rid of the Mentors under ANY pretext would be a good thing.)

The Lindens created a monster with the Mentors group. It is sprawling and unwieldy and just way too big at 3312 members, probably 1/3 or more who don't even log on (those who are members should check the log-in dates). SL does not need 3,312 of anything like that. 300 of them could do the job adequately. There's also a Mentors' Apprentices Group of 99 now, and then there are just way too many back-logged applications.

No, there are just way, way too many people who want the rush of infusion to their vanity that comes with being seen with a title that means they are Linden-Approved and also Helping Newbies, which enables them to enhance their reputation in this reputation-driven game, and also enables them to flog their content or services by handing out freebies or landmarks to their store. It's a terrible racket, and I think the Lindens should just go the route of the Central Asian dictator and just sell positions in the Mentors group, like some countries sell positions in the oil and gas ministry -- pay $50,000 Lindens to enter this group, and goof off and flog your wares and lure newbies to your club as much as you wish -- you're happy, we're happy, everybody gets paid.

But, since that would be really declaring failure, the Lindens are now in this God-awful process of trying to tame this beast. They've put a freeze on new memberships, but they can't seem to bite the bullet (after contemplating said bullet) of combing through existing members and eliminating the slackers and floggers. They talked vaguely of a "test" for mentors, but it's hard to understand what that would involve. It will inevitably be unfair to those actual dedicated and selfless Mentors who do exist, but maybe are poor test-takers, due to fright, lack of English, or lack of writing skills, even if they know things visually, and it will inevitably reward those Mentors who like to go swashbuckling through the welcome areas sporting the groovy title as a means of getting business, getting laid, settling scores. Sigh. Fact: Shaun Altman, with all his dubious stock market machinations and his rampant griefing of me and tenants, remains a Mentor. And I could cite a number of such cases, as could we all.

The smart thing to do would be to close down the Mentors completely as having, er, "overfulfilled their norm" and served their historical purpose, and open up a new streamlined group of no more than 300 (my God, there aren't even that many Lindens themselves). But, since these thousands of people represent hours, tier, loyalty, the Lindens, just like the tsars of all the Russias before them, cannot shut down these nobility and secret agent classes without harming the state itself. Or so they think.

There are other ways they might tame this beast -- insisting on hours logged of actual service, automatically expelling anyone who hasn't logged into the group for 30 days, spot-checking to remove those hawking their wares, diligently and quickly removing anyone who has had a disciplinary action, etc. But that's more administrative work for Lindens -- and the whole idea of this discredited concept is to "help".

I marvel that the "Greeters" are somehow going to be "better" than the Mentors. Huh? I think Greeters are just lower-class people, who don't have the building and scripting credentials that the Lindens so love which originally caused them to bestow the Mentor title. And basically -- again the problem of how to set up this experiment -- the issue here is not WHO is performing the function (which is misleadingly being reported) but THE FUNCTION ITSELF. By having the Greeters take people on tours of various business or club or shopping sims, they get them to "stick". Or so it seems.

Of course, there is a brand of newbie -- I know I was one, and you, dear reader, are another one -- who hates over-solicitous shopkeepers who ask "May I help you?" and trail you around the store, telling you this dress "would look nice on". You don't want a dumb tour, you want to be left alone to make your way on your own, or go to your colleagues or friends who will give you a "home stay" where you will be better cared for.

As I've been saying for years, it's like the refugee and IDP business -- home stays always produce better results for people and they adapt faster and are better positioned in the long run. If there was a way to organize and reward home stays, retention would improve. Yet this is very difficult to administer in a gaming and racketeering environment. Those bonuses for signing up members were gamed and alted and racketeered, and I think they have retired them.

I guess what I'd like to see the Lindens really do with this A/B stuff is the following:

1. Measure the retention level, as defined by log-on and user hours after 30 days and after 60 days if taken to resident-made infohubs versus anything else that is simply random. Some of these infohubs have presence of volunteers; others only occasionally, but they contain more friending and information capacity that the average dull welcome area, where there is more griefing, so they may show a tick upwards.

2. Measure the retention level from tours, versus nothing -- tell mentors and greeters to stay off some areas where there are just notecards and stations.

3. Measure the retention level from the customized business API use, i.e. measure the uptake to Adam Zaius' Azure Islands and Intlibber's Intlibostan, who are some of the privileged API users. (My recollection is that these customized orientation centers didn't do any better, really, the reports back a year or ago or something indicated this same uptick of a few percentage points, but it wasn't really impressive).

4. Measure the retention level from those using this new SLURL thing, which is ostensibly about people coming from websites from universities or pre-existing social networks or interest groups into SL directly into some specific venue or activity.

5. Junk the whole idea of measuring retention through the window of orientation island and mentoring and greeting. Drive the process of adaptation in a different way, by enabling people to sign up for experiences earlier in the process and go there, there could even be a dynamically loading events list or selected events that might be interesting, and constantly rotating venues, classes, places to explore. Naturally, people bitch about the Events list as containing too many sales of land and dresses, but there aren't as many as you think, and you can even leave them in to see if in fact newbies go to those fake events! Why not! You have to be impartial in these studies. The website cries out for change. It has had the same dumb people on it for years -- they need to be retired in the worse way, and the site should really dynamically change, as it did at once point with user-sent pictures.

6. Ads in the welcome areas -- portals, teleport boards. Yes, I've said this a million times, and now a million and one.

Comments

[Melissa Yeuxdoux]

"You know how they have these things called 'double-bind studies'..."

Is that where the subjects are damned if they do, and damned if they don't?

[Sean Williams]

The difference between a Mentor and a Greeter?

The Mentors are supposed to aid new and old users alike. A general purpose member of the group.

Greeters are most often to be found helping the newest and month old users with the focus on the newest of the new.

I hope that clears something up at the very least

[Ann Otoole]

Perhaps Linden Lab needs to seriously evaluate exactly why mentors and greeters are even needed. There are no mentors and greeters elsewhere. Well there is an old guy at wal mart that must have the brownie points to have that job but really all the wal mart "greeter" realy represents is a member of the loss prevention team to search people that set off the alarm as they exit.

So an efficient company with a good product that real people can use easily and figure out does not in any way form or fashion need "mentors" or "greeters".

Try explaining how to adjust a necklace to fit properly to a new resident sometime. There is no way this is going to come easily so the software needs to be determining where the attachment is intended to sit on the shape based on how the creator positioned it.

But that is hard to do. Not a chance of that ever happening in a company using the "tao".

[Gareth Nelson]

"This concept of the experimenter affecting the experiment is of course called the Heisenberg Uncertainty principle, and Niels Bohr also wrote about it"

Quantum physics fail.
Heisenberg's uncertainity principle simply states you can not know both the location and the velocity of a particle at the same time.
http://en.wikipedia.org/wiki/Quantum_indeterminacy

[Gareth Nelson]

"You know how they have these things called 'double-bind studies'..."

Is that where the subjects are damned if they do, and damned if they don't?

No, that's where neither the subjects or the researchers know who's on placebo and who's on the real thing.

[Jane2 McMahon]

Hmmm...when I taught university, I always wanted to teach first year students, and when I put on my Mentor hat, I want to greet newcomers. Likely it's for the same reason...all the smart, talented ones can go for the "advanced" and teach everyone their own fabulous skills...I just want to help whoever shows up to stay and check SL out furhter. If that makes me a lower-class drone, great!

[ichabod Antfarm]

Since you cite Wikipedia as an authority, Gareth, I will quote for you the first sentence of the article on the Heisenberg Uncertainty Principle:

"In quantum physics, the Heisenberg uncertainty principle states that locating a particle in a small region of space makes the momentum of the particle uncertain; and conversely, that measuring the momentum of a particle precisely makes the position uncertain."

See that word "measure" glaring out from that sentence? It is that measuring which is precisely the effect of the observer on the event being observed. Read further in the article. I will quote again for you:

"The uncertainty principle is related to the observer effect, with which it is often conflated"

So, although, Prok might have been more accurate to refer to the observer effect, his point still stands and his use of the uncertainty principle is hardly the case of "FAIL" you thought it was.

As for your second comment, you do realize that Melissa was trying to make a funny because Prokofy made a typo and spoke of "double bind" studies rather than "double blind" studies? As a nitpick, I thought you would have sunk your teeth into that one.

[Darien Caldwell]

While I've thought of Lindens as many things, Scientist has ever been one of them.

I applaud them trying to find new techniques to retain new users, but if what you say is true, They aren't likely to come up with meaningful results.

It can also be analogous to debugging software, something maybe some can relate to more. When attempting to discern the issue, one can change things and observe what the behavior of the code is after the changes. However, if you change 3 different parts of the code, you can't say for certain which change caused the change in behavior. It's important to only do one change at a time, or face the prospect of backtracking and trying to find out what change was the relevant one.

Tight controls on conditions is crucial.

As for Mentors, I have to agree, it seems they give that title to just about anyone. It almost always seems everytime I get someone in my store blasting gestures and making a particle poofs all over the place, When I check out their profile, yep, a Mentor. If there ever ends up being a way to ban by group, I know which will be the first on *that* list.

[Darien Caldwell]

"While I've thought of Lindens as many things, Scientist has never been one of them." - Fixed :p

[Melissa Yeuxdoux]

Actually, that's a double BLIND study. I guess I should've put a [sic!] in the quote.

[Ciaran Laval]

I hate reading your blog when I get home from the pub because I'm just gonna have to say how the fuck can a greeter be considered more important than a mentor? Greeters say hi, mentors do a shitload more. That's how it works everywhere!

Theser are volunteer positions right? Some people are going to want to self inflate themselves with a tag, it's not organised, it needs organising, how can this be a hard concept to grasp, it's this tao bullshit on a lazier scale.

[Yumi Murakami]

Hmm.... As I remember it, greeters were supposed to be the volunteers who met new folks in the WA and took them on SL tours. Proactive, while mentors were traditionally reactive. And, ironically, officially sanctioned to do what Prok complains that Mentors do - promote individual locations. In fact, they had to do a thing called Greeter Experience - basically the itinerary for the tour they gave - and have it approved by the Lindens..

[Prokofy Neva]

Actually, ichabod, I know all about this discussion, as it was all discussed ad nauseum in university and later in life. And Wikipedia is misleading, and you can't take Wikipedia or its readers as "authorities" on this matter.

Read this site, for example:
http://www.overcomingbias.com/2008/04/heisenberg.html

I've listened to wave physicists describe this and indeed it IS about the literal affect of the observer on the observed when measuring -- and frankly, you can't separate observing from measuring in this regard.

"Wikipedia, and at least some physics textbooks, claim that it is misleading to ascribe Heisenberg effects to an "observer effect", that is, perturbing interactions between the measuring apparatus and the measured system:

'Sometimes it is a failure to measure the particle that produces the disturbance. For example, if a perfect photographic film contains a small hole, and an incident photon is not observed, then its momentum becomes uncertain by a large amount. By not observing the photon, we discover that it went through the hole.'

However, the most technical treatment I've actually read was by Feynman, and Feynman seemed to be saying that, whenever measuring the position of a particle increases the spread of its momentum, the measuring apparatus must be delivering enough of a "kick" to the particle to account for the change.'

In other words, Feynman seemed to assert that the decoherence perspective actually was dual to the observer-effect perspective - that an interaction which produced decoherence would always be able to physically account for any resulting perturbation of the particle."

I'm talking about that very kick -- and BTW Bohr was, too.

[Prokofy Neva]

I find it loathsome that the Lindens would actually grant Greeters permission to flog various venues -- a kind of guided inworld Showcase. THAT IS AWFUL. JUST AWFUL. The market really should be taking care of this, not socialism. The Lindens need to stop all this fucking feting! My God, they only raised up retention a mere 3 percent by having their pets flog their other pets -- could they STOP DOING IT THIS DUMBASS WAY and allow people to buy ads in the welcome areas and take it from there? They can be PG; they can be rotated; but the point is, they will be ads that people have motivation to take care of customers because they get paid by the business itself to do so!

Nothing else is working -- when will they finally be ready to do that?

and yes, duh, it's a typo there, should be "double *blind*" but frankly, newbies find themselves in a double BIND as well, as they get either Greeters flogging their wares and Linden-pre-cleared and pre-sanitized venues, which are inevitably boring, or they get Mentors hawking their own stores.

Imagine if you ran Grand Central Station this way, and New York City's Time Square this way...

[Prokofy Neva]

I don't understand why Mentors, who I thought were by far the more feted and skilled group, would be overthrown for Greeters, which seemed less skilled, but I can only conclude this: the Mentors got too big for their britches and too out of control, and the Lindens needed people who could take orders better. That's my hypothesis. But I don't know enough about it.

Basically, my premise about the A/B experiment is simple: if you are testing the effect of a greeting program, the "B" that contrasts to it has to be "nothing" not other islands where "Greeters/Mentors" do their "usual thing". It's just too chaotic and not clean enough of a contrast.

And if you wish to compare "Greeters" and "Mentors" you have to have very clear-cut functions that both are doing, such as to be able to say "Mentors aren't needed". If you can't control the venue of "B" such as to understand what the Mentors are really doing "when they do what they usually do," then you can't really have a fair experiment.

[ichabod Antfarm]

That was a great article you linked to, Prokofy, "the Heisenberg Student Confusion Principle" says it in spades. I remembered enough of the chapters on quantum physics in Penrose's Emperor's New Mind to know what effect you were referring to and that Gareth had grossly over-simplified the concept.

As far as Wikipedia goes, I laughed when I read the article on Second Life and learned that Stephen Colbert had called Wikipedia "Second Life for Corporations"

[Yumi Murakami]

Looking at the meeting transcript, it seems that the Lindens are actually just going to create an entirely new and unrelated group called "Greeters", who are mentors who can enter the Prelude, as opposed to ones that can't.

I agree with you that "feting" individual locations is bad, but just advertising in the WA wouldn't compare to the old Greeter Experiences, which would give the newbie a responsive human to talk to, the opportunity to fly around and explore the world generally (this was still TeleHub days, so the Greeter would be flying them out from the hub as part of the tour, or maybe even using a vehicle) as well as a shortcut to interesting places. Sadly I can't think of any way to "tour" a user-created world without the places visited being created by particular users who benefit from the fetedness..

[Prokofy Neva]

Yumi, there shouldn't be created this separate group. Greeters aren't what are needed, nor should some resident venues be privileged over others -- the Lindens need to stay out of constantly feting only certain content, and let the market decide, and people decide by their free will, when given information. That's why we have what we call "advertising".

Those who advertise in welcome areas to attract newbies will have the staff to handle them -- or they won't get the business. And some venues don't need such hand-holding.

Police in the welcome area are for more important, to get rid of all the annoyances, infestations, and outright griefers. And that function should not be turned over to residents, who inevitably are biased and carry grudges and settle scores, more than the Lindens, anyway. One full-time paid, benefitted Linden circuit riding the hubs could accomplish wonders; two would really make a dent; three would probably solve the problem forever.