Onboarding both new and experienced programmers well

[kytrinyx]

The onboarding used to be horribly, absolutely, utterly broken. It was incomprehensible, and full of dead ends. It's no longer completely broken, but it's still not great.

I'm opening this issue in the hopes of finding ways of further improving the onboarding, including the option of scrapping what we have and trying something completely different.

When we were doing user research we learned that:

• experienced programmers were likely to successfully onboard, no matter how terrible the user experience was. They needed a quick overview, a link to download the client, and reassurance that the thing they were installing on their computer was not going to do nefarious things.
• newbies were thrown by almost everything. Jargon was intimidating. They weren't familiar with the command-line at all, they didn't necessarily know what GitHub was and why the hell Exercism had anything to do with that, and they'd end up on some random page to download the CLI and not understand where they were or what was supposed to happen next.

Also, it seemed like too much explanation/instructions frustrated experienced users, who basically knew the details, just needed to get it done.

I am worried that giving the same instructions to both groups of programmers. If we cater to newbies, the experienced programmers risk thinking that this is not for them, and being frustrated with the unholy level of detail. If we cater to experienced programmers, then newbies are likely to feel stupid and give up.

So as a first step in trying to address the broken onboarding, I decided to have two "how it works" pages. This might not be the best solution, but it did address some of the pain-points that we found when doing the user research.

@siakaramalegos noted in a discussion elsewhere:

I sometimes wonder if having separate sets of instructions for newbies vs experienced programmers helps. For one, I think it reinforces a fallacy that there is some high bar that is difficult to cross to become an "experienced programmer". When is that bar crossed? Secondly, I sometimes wonder if more experienced programmers are better at following directions or not. I think maybe they can troubleshoot bugs better, but not necessarily follow directions better.

These are great observations. I think that the biggest bar is context. Experienced programmers have a lot of it, newer programmers usually don't. As a result, I expect that one could give very short, compact instructions to experienced programmers, but not to newer programmers.

My impression is that the biggest barrier in all of this for new programmers is understanding PATH, though at the moment we aren't even really addressing that separately.

Another great comment from @siakaramalegos:

it might be even better if someone good at animations just created a few short (<1 min) videos explaining what exercism is and how it works/how to use it.

I'd love to have short video/animations.

Someone suggested using animated GIFs, and I have a problem with those, personally, because they are always moving and I can't think while they're in view. I also become physically unwell (I get the same thing with sites that have videos as the background to their homepages).

I have been thinking about how we could make an interactive guide to simplify the "CLI installation" portion of the onboarding. This is described in detail here: exercism/exercism#2535

[siakaramalegos]

It looks like a first-pass auto-detection of OS could be done with window.navigator.platform. There is also an oscpu property, but I couldn't get it to work in my browser.
https://developer.mozilla.org/en-US/docs/Web/API/NavigatorID/platform
https://developer.mozilla.org/en-US/docs/Web/API/Navigator

If you had maybe tabs on the same page for each OS, it might be pretty easy to use JS to autodetect OS and select the potential best tab. I would still put a sentence saying something like "We think you're using a Mac - if not, select the correct tab for your OS."

[kytrinyx]

Yepp, that sounds like a good approach. In fact, this would probably be a pretty low-effort first pass at reworking the CLI page on exercism.

[siakaramalegos]

Also, apparently there is a Ruby gem if you wanted to stick to Ruby - also does all the Regex to parse: https://github.com/fnando/browser

And a node package: https://github.com/ded/bowser

[kytrinyx]

I think that either way would be fine, so long as we make sure that if we don't have the information we provide a reasonable message/default (I don't know what this would look like with a text-only browser like links or lynx, or with a screen-reader for example.

[kotp]

A solution to the animated gifs, and a gentle introduction to the terminal... Not necessarily this ascicast, but the ways in which it can be applied.

For the places where the actual script is allowed, it has a play, pause, copy and paste functionality. All the good things ,none of the motion sickness and distraction inducing things.

[kytrinyx]

Nice! Yeah, if we can get gifs that let us play/pause/copy and paste, that's great.

I also think it would be lovely to have video that has animated diagrams for explaining at a high-level what exercism is about.

[kytrinyx]

Ref: #34

[kytrinyx]

Reference: http://theelearningcoach.com/learning/novice-versus-expert-design-strategies/

[troubalex]

Sorry for airdropping into the discussion, I just came across this issue after talking to @kytrinyx Bear with me, if I misunderstand or lack context. 🙇

Here are some suggestions and ideas, based on my experience designing for techies.

The most efficient way for experienced programmers is to give them a big download button, based on the OS they're on, provide a link to all the other packages, and tell them where to find the configuration instructions. It's probably fair to assume, they either know how to add stuff to their PATH, or at least know how to google for it. Ideally, the download button would actually start the download, rather than taking folks to Github, but I don't know how feasible that is without manually updating the link with every release. Which might be worth it.

To avoid a certain "real-developers-go-here"-effect which could scare the newbies, this could be achieved through a sidebar that holds the button and some kind of navigation for the different steps involved. Then experts can download their package and jump straight to the configuration step.

For new-beginners, it might make sense to create dedicated areas for installation instructions, e.g. "Install exercsim on Windows". That way, it is easier to divide up content, and provide the sidebar navigation which lets user jump to specific steps in the process at all times. Another side-effect could be a certain feeling of achievement at the end of each step. You might even provide a check-list showing progress.

It could also be a good idea to add (annotated) screenshots, especially for the Windows instructions, as finding the command line is not as straight forward as on other OSes. They are much faster and easier to make than screencasts, but still quite reassuring if you don't actually know what you're doing.

Then, I'd suggest simplifying the content for newbies even further. Right now, the instructions try to cover all cases which might be overwhelming. Is it possible to find one happy path, and move everything else into the troubleshooting section? For example, you could say that installation via Homebrew is recommended, and tell people to install that first if they don't already have it. Then you could link to a section within the troubleshooting guide that explains how to install the binary manually if you can't or don't want to use Homebrew.

I suspect that people get stuck with different problems anyway. The more experienced ones will google for a solution, but the other ones might need help. I saw in the Readme that there is a chat on Gitter, maybe consider making that more prominent inside the install guides. I didn't see it anywhere at least.

I would probably also move the biggest chunk of the detailed explanation which is now on the "how-it-works"-pages, into some kind of help section. That way, you can focus the copy on the "marketing-pages" on explaining your selling points for each target group. Currently, it's a mix of documentation, contextual explanation such as what Github is, and a feature overview. And there is a lot of text. 😉

I suspect that the newbie-version will need more explanatory copy than the other ones but that can be solved in different ways, e.g through well-phrased micro-copy, and/or an FAQ section.

Finally, what about moving all the instructions behind the sign-up? I don't remember what the first view looked like right after signing up, maybe it's already there. I imagine that what people most care about after signing up, is getting started. The welcome screen should therefore be the first one in the installation process. Having the user log in first also gives you the opportunity to provide the real API key inside the commands for easy copy-pasting.

I hope this makes sense, I'll happily draw some quick sketches if that helps. And sorry again for the huge brain-dump coming out of nowhere.

[troubalex]

PS: Take a look at this, in case you haven't already 👉 http://www.useronboard.com/

[kytrinyx]

@troubalex This is brilliant, thanks so much for the detailed suggestions!

[ghost]

Notes of issues whilst going through the CLI installation instructions:

1. Everyone needs to create an account before downloading the CLI. It's a separate piece of work to making those CLI instructions easier, but it would be great to have a numbered step through the process of onboarding, such as what keystonejs.com do.

2. I'm wondering if Command Line Help after getting someone to use the command line is too little too late, and if there's somewhere else that better belongs - such as a "Before you get started: Requirements" section - which can mention command line, package requirements for each operating system, choosing a language to work on, etc.

Getting Comfortable on the Command-Line

If the command-line feels foreign and intimidating to you, go work through the excellent tutorial
Learn Enough Command-Line to be Dangerous.

3. Submitting Solutions within the install CLI
   There's too much thinking that needs to go on here in order to keep this information with CLI instructions. I'm wondering if this information is best kept in the languages themselves (which it looks to be, so lets remove it from the CLI instructions if that's the case). Reason being, there's too much steps removed and thinking going on here in order for this to work.

Submitting Solutions

Use the submit command with the file you want to submit:

exercism submit path/to/file

To add a comment to your solution, use the --comment flag:

exercism submit path/to/file --comment 'Implementing it this way because...'

This is a great place to say what you were thinking, what was interesting about
the exercise, what you got stuck on, what surprised you, what trade-offs you
made, etc.

[kytrinyx]

Everyone needs to create an account before downloading the CLI.

No, not really. You need it before you can be recognized, but you could easily download the CLI and exercism fetch go clock or whatever without being authorized. You just can't submit back.

That said, I don't mind if we simplify and pretend that you do need an API key first.

[kytrinyx]

There's too much thinking that needs to go on here in order to keep this information with CLI instructions.

Yeah, I think you're probably right that just getting the command-line tool installed and configured is good here; actually using the CLI to interact with exercism (fetch + submit) should be documented separately.

[ghost]

There's a great article on onboarding users:
https://uxmag.com/articles/onboarding-designing-welcoming-first-experiences
One of the things that seems to be particularly important to exercism is how to get people new to the site involved in the community - so if we can show from the beginning how to do that, like Tumblr does, that could be a good start:

"One of the biggest differentiators for Tumblr is its highly active community, but until a new user is connected to several other bloggers, she might not truly comprehend its magnitude. Tumblr overcomes this by encouraging users to find their friends early on, in a variety of methods. A user can scan for contacts in an existing email address book, or follow people under Staff Picks, Music, or Art and Artists. By observing the usage patterns of these popular and highly active users, new users will most quickly understand how to use Tumblr to their advantage, nearly guaranteeing more frequent use of the site."

I don't know about finding friends or contacts - but is there someone/something else they could follow/do? When they pick a language, what could we show them that would help them get involved?

[kytrinyx]

To my mind, the way to get involved on Exercism is to give feedback about someone's solution.

[jmbradnan]

I am new to Exercism and not an experienced programmer. I decided to give Exercism a try and first off, its awesome! I have a suggestion for content and not sure if this is the right forum or it should go elsewhere. I had no difficulty getting the CLI installed, I did get stuck with what to do once I had the test cs file - how did I use it in my project, what did I need in place to get NUnit working etc. It actually took me a while to get things working even with searching online for help. I can imagine a brief how-to guide here, although it gets a little IDE specific so would have to be careful not to get too into the weeds trying to cover every tool. If this has already been suggested, lmk I wasn't able to locate an issue tracking this, but may not be looking in the right location.

[rpottsoh]

@jmbradnan welcome to Exercism.io! The C# track has pretty good looking documentation, it is available here. Also the C# track offers support via gitter here. I am not affiliated with that track, but the guys that maintain it know their stuff and are very helpful.

[kytrinyx]

This is being tracked as part of the nextercism redesign (see #113 and #154).