Results
Participants
Eight participants were given usernames and passwords to log in to the online community space. They were asked to log in as frequently as they felt they wanted to, and were told that they were user-testing a new website. The true nature of the study was not revealed to them until after the research was completed.
Participants in the study were mostly former students of the Advanced Diploma of Multimedia at Victoria University. Some of the participants had previously studied together, while other participants did not know anyone else in the group.
Participants were not asked to sign a 'conditions of use' contract before logging in, making the community largely self-regulating. As they were aware that they were participating in a research project and their activity was being monitored, no socially unacceptable comments or posts were recorded.
There were equal numbers of male and female participants. They ranged in age from 20 to 36. The majority of participants reported that they were frequent users of the Internet, spending some time online every day. In terms of online chat, the majority of participants described themselves as casual users. Only one participant described herself as 'not really interested' in online chat.
The majority of participants reported that they had two or more active accounts with social networking sites such as Facebook and MySpace. As most of these sites display user information prominently in the interface, the participants were able to anticipate where to look for the social proxy and had a keen understanding of how to interpret the information it displayed. Some participants had very specific expectations of how the social proxy should look and feel. One participant posted a 'feature wish lists' of how he'd like his personal information displayed, including where he felt it might best fit in the flow of information across the site:
"...an added feature would be to place the profile roll-over in the footer area below the people and activities area. This could stay there when the profile is clicked or have the new profile appear when chatting. Kind of like looking face to face when talking."The above statement is indicative of the high level of new media literacy among participants.
First Research Cycle
Participants were each contacted separately and given user names and passwords to log in to the site. Within the first twenty four hours of the site being live, all of the participants had logged in and experimented with the various features of the space. Brief conversations took place between participants in the 'chat' area, and the 'share' section of the site became highly active. Participants created new 'share' entries and commented on each other's entries, with the most popular topics revolving around the various technical glitches and bugs each had discovered and exploited.
As anticipated by Whittaker, one member of the group was responsible for a significant amount of activity on the site, while two members of the group could be considered 'lurkers' as they barely registered any activity at all. One of the participants discovered a means by which to artificially inflate the number of minutes of activity recorded in his profile. In a little under three days, the participant had recorded over nine hundred minutes of activity, well above the average for other participants. One of Erickson's claims for social proxies states that deception should be allowed, and so nothing was done to curtail or otherwise address the participant's behaviour. The participant was by far the most legitimately active member on the site, despite his deception.
At the end of the first cycle, a survey was distributed and returned complete by five of the eight participants. Two participants said they'd logged in every night to check for new content or chat with other participants, while the other three checked every second night.
Three out of five participants agreed to the statement: 'I want to be seen as an active participant.'
The majority of participants reported that they did not frequently change their own profile information. Significantly, three of the participants reported that they often looked at the profile information of other participants. This suggests an awareness on their part of the social proxy displayed in each profile.
Usage data showed an upward trend in activity in the initial three days of the first research cycle which leveled out and then dropped slightly. On average, participants recorded approximately 334 minutes of activity and posted approximately 11 chat messages and 3 'share' entries or comment over the course of the research cycle. Two of the participants were responsible for 94% of the activity on the site, while the least active member was responsible for less than one percent of activity. These results do not factor out the artificial inflation of the statistics of one of the participants, as described above.
For one participant, perceived activity as represented by a social proxy was crucial for engagement, and the following suggestion was offered as an enhancement of the feature:
"Having not "chatted" before I was surprised to get a kick out of it....Unfortunately many of the other participants don't seem to be online when I am so my call-outs have gone unheard. Perhaps an indicator of how much activity happens at different times of day would increase my chance of connecting."The data suggests that the social proxy was being looked at and relied upon as an indication of participant activity and engagement.
Second Research Cycle
In the second research cycle the social proxy was removed without announcement or forewarning and usage data was collected silently.
An email questionnaire was distributed to one of the participants shortly after the social proxy was removed. To the question, 'How did you feel when your stats disappeared in your profile and the profile of others?' the participant replied:
"Awful! Now I can't check up on the others to see if they've been talking...I looked at their profiles a lot, and mine also, to find out if I was being a good guinea pig or not. Before you took it away it was hard to get online at the same time as the others, but at least I knew that something was going on."Clearly the perception of activity on the site was important to the participant, and without the social proxy it became difficult for her to gauge how active she should be. Consequently, the participant's activity levels dropped. Before the removal of the social proxy the participant had submitted 17 chat messages, while after the removal she submitted only 4 chat messages in the same time period.
Activity dropped significantly for all participants after the removal of the social proxy. Participants averaged only 2 chat messages per person and the average amount of active logged-in time fell by more than two thirds. The participant who had 'gamed' the social proxy in the first research cycle appeared to have 'gamed' it in the second, as well, but less so, and roughly in proportion to the drop in activity of the other participants.
This drop in activity was also reflected in the responses to the second Likert-scale survey that came back at the end of the research cycle. Only two participants out of eight filled out the survey. Surprisingly, the participants that did respond reported that they checked the site once every second night, and that they still regularly looked at the profile information of other participants. This indicates that while 'lurking' behaviour continued, tangible activity in the form of posting messages slowed to a crawl.
In the absence of the social proxy, one participant attempted to fill the void with his own suggestions as to how the site might work better to ensure engagement:
"I would've liked the fellow chatters in the group to be displayed via an icon tag cloud. Preferably one that was clickable to delve further. I'm a visual interface kinda guy. Like the way that tags are displayed on blogs, the avatars or simply their names could appear. If you wanted to get really tricky you could look into adjusting the size of the name tag according to activity. With a nice layout and good spacing of the name tags it could called 'clouding' (sic)."The data appears to point to a significant overall drop in tangible activity levels for all participants after the social proxy was removed from the site.
To log in and have a look at the online chat tool, please click here. You'll need a username and password. Email me to get one.