fastn tutor

[amitu]

Status: WIP, Owner: @amitu

• tutor mode (cli argument and fastn_core::utils::is_tutor() function)
• shutdown api: we will use signal instead
• pwd api
• tutor config & processor
• config refactor
• make-current api
• stop-current api
• tutor.js
• tutor ui

keywords: fastling, rustling

fastn tutor to help learn fastn.

fastn tutor

This will expect current folder to be checkout of the workshop. It will launch a UI, which will show status of what all steps are marked as done.

Tutor UI

It will get the list of steps by scanning the workshop, and finding the first two levels (first level is workshop name, eg a-workshop, and second is a step in the workshop, eg a-workshop/01-hello-world.

The tutor ui will also have button to mark a step as done, which automatically moves to next step. Next to any other step there is a run button as well, which makes that step the current step.

Based on current step the tutor ui also shows some instructions on how to finish the step. This is read from the workshop folder itself.

Current Folder

Till fastn can download the workshop, it will be assumed the current folder is workshop folder, and workshop folder contains the fastn tutor related package as well.

State

fastn tutor will store the state in ~/.fastn/tutor.json file.

struct TutorState {
    done: Vec<String>,
    current: String,
}

When the file is created, done will be empty and current would be the first step.

Both will contain names like a-website/01-hello-world.

Tutor Mode

fastn tutor will run fastn server in the folder corresponding to current. When fastn starts like that it will run with fastn serve --tutor as the CLI argument.

Fixed Port

In the tutor mode, the port cli argument will be ignored, and it will always run on port 10345.

/-/shutdown/

In tutor mode request to this URL will stop the server. This will be used by tutor to stop the server and start a new server when learner moves from one step to next.

/-/pwd/

This URL will return the pwd, which will be used by tutor to check to ensure the right server is running. If not, it will shut it down and start another fastn on right port.

tutor.js

When running in tutor mode fastn serve will auto inject a tutor.js (this can be done using --js cli argument.

tutor.js will map t shortcut, which will open a popup, which will show tutor UI in an iframe.

Downloads Github Workshop - LATER

fastn tutor will checkout workshop if it is not yet there.

[amitu]

Status update

I have merged two PRs so far (#1415, #1420) and a third PR is ongoing (#1434).

Tutor Page

We have created a new command fastn tutor, which expects the current folder to contain a .tutor directory, and runs fastn server on port 2000. It is supposed to look like this:

This is how it looks like right now:

The iframe on left is not meant to be there.

The data for tutorial is in the .tutor directory.

We also have a file ~/.fastn/tutor.json which contains done tutorials and the one that is currently active.

#[derive(Debug, Default, serde::Serialize, serde::Deserialize)]
struct TutorState {
    done: Vec<String>,
    current: String,
}

If the file is not present use TutorState::default().

Tutorial Page

There is a concept of a current tutorial. When a tutorial is made current on Tutor Page, we start a new fastn server, in the folder corresponding to the selected tutorial, and show this page.

We inject a tutor.js file to ensure the left side iframe is added in any page rendered by the second server.

[amitu]

The Two fastn servers in tutor mode

These are some notes on what I am going to do soon.

Why do we need two servers?

When working through tutorial we expect learner to manually cd into the the folder corresponding to tutorial and start the server there. And they have to stop the old server. This leads to many errors, sometimes people forget to switch and they keep seeing old UI on reload, while they are modifying code in the next folder. Sometimes they accidentally start fastn in both folders, and second one starts on port 8001, since first one is still running on port 8000, and browser has 8000 open, so they end up seeing output of wrong fastn server.

Further, there is a lot of switching, there is editor, there are instruction on our website, and there is output in browser. So we want to show the instruction and output side by side.

In future we may even include some js based editor, and user can code, learn and preview all in single tab in browser.

How does fastn tutor work?

fastn tutor has to run in workshop checkout. That folder contains .tutor folder, which contains instructions for each workshop/tutorial.

When we start tutor using fastn tutor command, we cd into .tutor folder, and then run fastn server on port 2000. This fastn server acts as the controller.

In the UI that is shown above, there is a play button next to each tutorial, when that button is pressed the controller fastn starts a new instance of fastn server, in the right folder. The slave fastn always listens on port 2001, so there is no port confusion.

Proxying

When fastn slave is not running, all HTTP requests to port 2000 is handled by fastn controller. When we start the slave, all non-tutor requests will get proxied from the controller to slave.

What are tutor requests? Any URL starting with /-/tutor/ is considered a tutor request, and it should always be served by fastn controller. Any other URL will be proxied to slave fastn.

Special handling on slave urls. The url of tutor is /-/tutor/ (any other URL will get proxied!), so if slave is not running, we redirect any request to /-/tutor/.

[amitu]

How does fastn controller know if slave is running?

Since slave can be either running or not running, how does the controller know? How does controller start the slave? And how does it proxy requests?

Do we really need two servers? It's a bit heavy handed approach. Can we not simply re-use the same server and call the serve or handle method?

Currently we have a global config, so if we have to servers running, we will need two config objects. Also the code is written with the assumption that all files are in current directory, and our two servers are supposed to serve from two different folders.

What if we add the working directory in config and make multiple instances of config?

Currently each route handler is responsible for creating config. This is not great in general, we are creating config all the time, which is rather expensive.

Re-using means config could be stale. The right way would be to watch folder for changes, and update config only on file change. Or do it prod/debug mode, in prod mode we do not expect the file system content to change, and we can re-use the config object.

Actually there is another reason to re-create config, as we keep some mutable fields on the config object, which contains data specific to current request. This has to be re-created. Right thing would be refactor config into more than one such, the dependency stuff, which is "heavy" can be re-used, and per-request stuff, which is cheaper, can be created on each request.

What to do?

[amitu]

Config Refactor

So I am finally biting the bullet and refactoring config into two structs:

#[derive(Debug, Clone)]
pub struct RequestConfig<'a> {
    pub named_parameters: Vec<(String, ftd::Value)>,
    pub extra_data: std::collections::BTreeMap<String, String>,
    pub current_document: Option<String>,
    pub dependencies_during_render: Vec<String>,
    pub request: &'a fastn_core::http::Request,
    pub config: &'a Config,
}

Every method that used to get Config, will not get RequestConfig. The PR is going to be full PITA. Right now I am getting 73 compilation errors. And this is the kind of change that can really but head against Rust's borrow checker. But I think we are on right track.

cc: @Arpita-Jaiswal

[amitu]

158 now!

[amitu]

After fixing 500 more errors we are at 89.

[amitu]

This is stuck in weird state: code is type checking, but test is not passing, nor are we able to do cargo install, it's taking forever to compile somehow.

PR: #1434

@Arpita-Jaiswal is taking a look at it for now.

[amitu]

After a lot of patient work @Arpita-Jaiswal managed to get it to compile etc now. Some unrelated ftd test is failing, works on my machine but is failing on Github. Will leave Arpita to figure it out.

[amitu]

There is also some font issue remaining.

[amitu]

So now that config refactor is almost done, still in PR, but we can continue back on tutor stuff. When we are serving, we need the the package current folder, earlier it used to be always pwd, but now we can make it anything.

Also since tutor is there to teach you all features of fastn, and some of those features include CLI arguments, like --external-js etc, which we read from CLI argument, we have to figure out how to make them also configurable after the server has started.

This is actually quite easy because we create a new config object on each request, and all the external-js etc we pass it to the config object. Where would we get these parameters?

Modes

fastn runs in three modes now:

Normal Mode

In this the default, the external-js stuff comes from CLI, and folder is current folder (we can make this configurable by CLI / env var also).

Tutor Mode

This is when we run fastn tutor. In this mode we have only one page, /-/tutor/ (and its media dependencies etc). The external-js etc is not configurable by end user.

We are in tutorial mode when fastn_core::tutor::is_tutor is true and fastn_core::tutor::tutorial is null.

Tutorial Mode

This is the when the learner has selected a tutorial.

We store the current tutorial folder in a variable fastn_core::tutor::tutorial: Option<fastn_core::tutor::Tutorial>. It can be made lazy static / mutex etc could be needed.

We have API like start tutorial and end tutorial, which basically mutate this variable.

We are in tutorial mode when fastn_core::tutor::is_tutor is true and fastn_core::tutor::tutorial is not null.

Tutorial Mode CLI flags

In tutorial mode we need possibly different set of CLI flags for each tutorial. To get this flag we are going to store them in .tutor folder, in the <tutorial-name>.ftd. We are going to store the flags as well in the Tutorial:

struct Tutorial {
    path: String,
    app_data: fastn_core::commands::serve::AppData,
}

.tutor/index.ftd will extract tutorial folder and the app_data from <tutorial-name>.ftd and send it to /-/tutor/start/?tutorial=<pwd>&external_js=<> (or maybe as a POST parameter).

[amitu]

Wrote some thoughts on configuration: https://github.com/orgs/fastn-stack/discussions/1451

[amitu]

So let's recap where I am: I have API's to set current tutorial (untested), our server can now serve the content of the current tutorial (untested), and a processor to show tutorial data, including current tutorial (also untested). I have not called those APIs yet or tested any of these features. And for that matter tests are not passing right now in the PR and fonts are not working.

So what do I do next?

Start Tutorial

We have to show the processor data in tutor.ftd file, and then call the start tutorial API. Once done all subsequent requests should be served by that tutorial.