For the complete list of panels, check out the main TwitchCon 2016 Panel Recap post.

VoD link: https://www.twitch.tv/twitchconfrankerz/v/92237135?t=06h00m10s
FrankerZ Theater Day One
Moderator: Dr_ewww
Participants: advntrcptlst, HorsesOnDemand

• The Twitch Science team’s mission is to help reach decisions about how Twitch products and programs should be. They collect data and supply these to people.
  • The user research team at Twitch aids in product development by helping teams understand user needs, motivations, and behaviors.
  • They work across the organization with product managers, designers, engineers, marketers, and product leaders.
  • They collect data by using research methods and working directly with Twitch stakeholders, viewers, broadcasters, and (game) developers.
• A typical research product development cycle involves the following:
  • Ideation
  • Prototyping
  • Development
  • Launch
• Everything on Twitch starts with an idea. Often these ideas come forth out of talks with viewers, broadcasters, and partners. Twitch has ideation workshops to think of new ideas for Twitch.
• A form of Twitch prototyping is paper prototypes. It’s cheap to make and an easy and quick way to get instant feedback.
• Twitch has a lab with all the different platforms that Twitch is used on to test the prototypes.
• The work doesn’t stop when the product is launched. Post-launch, the Science team partners with customer service and data teams. Collect information using surveys and use all these feedback to better the product or perhaps use them for a future product.
• The friends system is one of the bigger launches the Twitch science team worked on. One of the challenges was finding a way to discover friends without having to search for them. Twitch made recommendations for whom to befriend based on the following interactions:
  • whispers
  • chat mentions
  • follows
  • hosting
  • co-moderating
  • shared friends (post-launch)
• Goal is that people accept friend requests based on these recommendations, so the system was fine tuned to increase the acceptance rate.
• After the friends system was added the percentage of whispers between friends was measured and turned out to be really high, another indication that the recommendations worked.
• One of the reasons the friends system was added so that you can share who you watch and thus increase follower rates for those broadcasters. Between community members, it increased follower rates among streams followed and watched by friends.
• Experimentation is very useful for the Science team and should also be for viewers. When you’re playing games you kinda work on experimentation already by trying to do things that make you better at a game. Experimentation has 4 principals:
  • Pick something to improve
  • Make a change you think will help
  • Minimize other changes
  • Measure the result
• The ‘recently watched’ system is a feature that for instance helps with finding channels you’ve watched but for some reason didn’t follow or remember the name of. They experimented on this feature by trying to figure out if this would work with games too instead of just channels.
  • Goal: help people find something to watch
  • Change: show recent games
  • Minimize other changes: A/B test (compared two groups of people, one affected by the change, one not affected by it)
  • Measure: compare how much video people watch between control + treatment, this was measured by:
    • How much video people watch: people turned out to watch moderately less with the interface
    • People’s odds of returning to Twitch: Didn’t get affected much
    • How often people loaded the game directory: Was loaded quite often, but was more about novelty than being something useful.
    • How often people followed a channel: Was unchanged.
• All this helps to understand the products they design. They design something to make a change but they don’t always know if it has the desired effect.
• Try this at home, how to promote your channel:
  • Lot’s of streamers tweet before they go live, when they go live, and then stop. The panelist believes that tweeting throughout the stream will draw in more viewers.
    • Goal: Get more viewers!
    • Change: Tweet every hour while live
    • Minimize other changes: Stream normally for a week, then stream for a week with the new twitter change (it’s important to base this on a whole week, since different days have an impact on viewer numbers and so does the weekend versus the work week)
    • Measure: Compare how many viewers you get (and when you get them)
  • Decision time:
    • Did you see a change in your metric?
    • Do you think that change was the result of your new habit or something else
    • Make a decision!

• Another experiment to try at home that is being recommended is the auto hosting feature that just got released.