Url gets parsed wrong in development mode

[Waxolunist]

[EDIT: Being able to have relative references from CSS to assets work the same in dev and production is quite important]

I am using express together with socketstream. Besides that I am using bower for client dependency management.
I configured bower to download its contents into /client/code/system.

Bootstrap is also in system as bower dependency. I configured stylus to include the bootstrap css. Bootstrap loads its fonts (glyphicons) relative from the url ../fonts/fontfile.

That means, that requests in development go to /_serveDev/fonts/glyphicon.woff and in production mode to /assets/fonts/glyphicon.woff.

In development however the original url gets parsed as follows:

originalUrl: '/_serveDev/fonts/glyphicons-halflings-regular.woff',
  _parsedUrl: 
   { protocol: null,
     slashes: null,
     auth: null,
     host: null,
     port: null,
     hostname: null,
     hash: null,
     search: '?glyphicons-halflings-regular.woff',
     query: 'glyphicons-halflings-regular.woff',
     pathname: '/_serveDev/fonts',
     path: '/_serveDev/fonts?glyphicons-halflings-regular.woff',
     href: '/_serveDev/fonts?glyphicons-halflings-regular.woff' },

You see that the file name gets changed to a parameter without value.

Whereas in production mode the same request is as follows:

originalUrl: '/assets/fonts/glyphicons-halflings-regular.woff',
  _parsedUrl: 
   { protocol: null,
     slashes: null,
     auth: null,
     host: null,
     port: null,
     hostname: null,
     hash: null,
     search: null,
     query: null,
     pathname: '/assets/fonts/glyphicons-halflings-regular.woff',
     path: '/assets/fonts/glyphicons-halflings-regular.woff',
     href: '/assets/fonts/glyphicons-halflings-regular.woff' },

So in order to serve this file I have written following middleware:

var app = express();
app.get(/^\/(?:_serveDev|assets)\/fonts(?:\/(.+))?/, function(req, res){
  var file = req.params[0] || Object.keys(req.query)[0];
  res.sendfile(bowerpath + 'bootstrap/dist/fonts/' + file);
});

It's not a big issue, but why do requests to _serveDev get parsed different, than those to assets?

[RomanMinkin]

Hi @Waxolunist,

Fonts should be within /client/static folder, let say /client/static/fonts/
You also need to change path to all the fonts in all the css files to be relative to /client/static folder, for example:

@font-face {
  font-family: 'FontAwesome';
  src: url('/fonts/fontawesome-webfont.eot?v=4.0.2');
  src: url('/fonts/fontawesome-webfont.eot?v=4.0.2#iefix') format('embedded-opentype'), url('/fonts/fontawesome-webfont.woff?v=4.0.2') format('woff'), url('/fonts/fontawesome-webfont.ttf?v=4.0.2') format('truetype'), url('/fonts/fontawesome-webfont.svg?v=4.0.2#fontawesomeregular') format('svg');
  font-weight: normal;
  font-style: normal;
}

[Waxolunist]

Yes I know that, but that makes my bower components not updateable. I know at the moment a solution supported by socketstream is missing, but I am trying to work around these problems, when possible without changing the assets coming from dependencies.

I wanted just know why one time the request url gets parsed with the filename as parameter, the other time as it should be.

[RomanMinkin]

Got you, answering you question:

It's not a big issue, but why do requests to _serveDev get parsed different, than those to assets?

It because of internal middleware, which intercepts request before router's middleware: https://github.com/socketstream/socketstream/blob/master/lib/http/index.js#L55

To archive you goal we need to change the request/respond model in general, because it has been developed the way to serve assets from hard-defined/static directories.

I already was thinking about it already, especially how to make SS support Best Practice Recommendations for Angular App Structure

[RomanMinkin]

@Waxolunist, are you using Express? In this case you can try to add an additional static folder

app.use('/fonts', express.static(path.join(__dirname, 'bower_components/bootstrap/fonts')));

p.s. there is also should be a way to do it with default connect...

[Waxolunist]

Thanks for clarification.

Yes I am using express. I have written a small static middleware for serving the static fonts (see above). So it works.

However, when using a static folder, I have to change bootstrap and move files around. Which is not a lot in the case of bootstrap, but think of a more advanced widgetset library, which has a lot of sprites, images, fonts and dynamic paths.

Socketstream is great when it comes to initialize a project. It is very easy to ramp up and start. But when clients get more sophisticated, it starts getting difficult and it feels like fighting the framework, especially on the client side. Every small library is a pain to get to work. You have to move files around, change paths in well established libraries or configure those libraries to fit into the socketstream directories.
Socketstream is very opinionated about the client side. Being opinionated is not a bad thing at all. I love opinionated frameworks, because they give structure, but opinions should you not hold back.
For instance I can not easily use the power of r.js for optimizations. I can not declare dependencies (bootstrap depends on jquery, but I can not express this relationship). I can not use different formatters for the same file extension in different paths. Not able to use dependency managers for the client side. Not able to define multiple pre-processors for the same file. So imho socketstream does not get the client side right at all, even if so stated.
The preprocessors are bound to file extensions and paths in the client directory and are very unflexible. Libraries which should get browserified put in that directory, files with that file extension into that directory but not if a formatter should be excluded, and so on, but the days of going to a projects website and download single files are over.

On the server side a lot feels right, except, when it comes to serving static files, socketstream reads those files synchronously (I saw at least a lot of sync calls). But the rest is ok, especially the rpc or pub/sub parts.

I thought a lot about how to structure the client side. My requirements are not difficult:

• Dependency management with bower as it is
• Using pre-processors
• Chain pre-processors
• Cache the outcome of those pre-processors
• Save the outcome for production if needed
• Initialize socketstream on the client side

I looked at http://www.metalsmith.io/ and http://gulpjs.com/ and some other libraries and wanted to utilize one of those, because they both fulfil most of the above requirements, but I was not able to get it to work without breaking socketstream, too strong is the binding of the client to its directory structure and to weak my knowledge of the internals of socketstream.

Maybe you should provide a gulp interface for the client. Everything done on the client side (e.g. ss-stylus, ss-jade, browserify) can be done easily with gulp plugins already available.
Socketstream should therefore concentrate on the server side and provide a default gulp configuration which copies todays client side behaviour, so if not configured otherwise, the behaviour would be the same, which would allow for a smooth transition.
Advantages would be

• higher flexibility on the client side,
• less issues and PRs for those parts,
• less repositories to maintain
• a huge plugin eco system (400+ plugins), which will socketstream never reach,
• smaller code base.

Please, don't understand me wrong, I like socketstream and contributed some small code pieces already, but the restrictions on the client side, make it difficult for me to persuade others and more and more myself, to use it for further development. I like how it evolved in the last months, with a cleaner code base and better documentation, but it still lacks flexibility on the client side.

What's your opinion about transition the client side to an already established library and abandon the socketstream part?

[kulicuu]

This is very interesting.
I'm not really expert enough to respond in detail but hopefully others will.
Your recommendations on structuring the client side especially seem promising, but again, I'm not resident expert yet. I hope some others weigh in.

[RomanMinkin]

Hi @Waxolunist,

First of all thanks for righting that!

Sorry for delay. I see what you say. I'm facing the same issues right now, especially when i'm trying to implement recently released Best Practice Recommendations for Angular App Structure

So i do understand that eventually SS needs to support more abstract way for assets managing.

As may know we abandoned 0.4 version. But even v0.4 had the same assets delivery pattern as v0.3 has right now.

I have never work with both http://www.metalsmith.io/ and http://gulpjs.com/, so let me check them out.

it's could be workaround for v0.5, so do you think you would be able to contribute? Unfortunately there are only few people in the project right now who do contribute.

Let me know what do you think.

[Waxolunist]

I'd like to contribute, but I will not find any time in the next months, as I am starting a new job and my family will grow ;)
Is there any roadmap for the next versions?

[paulbjensen]

@Waxolunist I know the feeling (minus babies), congratulations.

I'm currently on holiday in Australia until Friday, I will catchup on this over the weekend.

[paulbjensen]

Hi @Waxolunist,

Sorry about the lengthy delay on getting back to this - I had a great holiday, followed by a very busy few weeks following (in fact I had to work towards the end of my holiday).

The points you raised are very valid - the client-side pipeline workflow is nice, but it's opinionated approach makes it difficult to work with package managers like bower and component.io.

I've recently used Gulp at work for a desktop application, and I think that it is a brilliant tool for client-side development - I think that there is potential for it to be a client-side replacement to SocketStream's client-side part.

In order to do that though, we need to take a good look at the framework, and begin the work of getting to the vision of 0.4, by way of refactoring 0.3.

I want to make it possible for SocketStream to be used in a more modular fashion - such as being able to make use of the RPC/PubSub features without requiring the client-side part, as well as being able to build the client-side assets without having to boot the whole app.

These are things I'm going to be looking at this week.

[essentialjs]

I think there are some thoughts in this thread that relate to "What is Socketstream". It would good to have an updated mission statement for Socketstream so people know what "They are getting".

To me r.js was an interesting solution that is appropriate in certain situations, but not the right one for public websites due to async requests nature. Browserify is good for that use case and should be the primary solution in Socketstream. We should actively discourage r.js, but we shouldn't actively support it as it is a very different concept.

We could refer to other peoples blogs/articles on the subject like:

http://esa-matti.suuronen.org/blog/2013/03/22/journey-from-requirejs-to-browserify/

[paulbjensen]

I agree that SocketStream needs an updated mission statement. I want SocketStream to be the best framework for creating Realtime Web Apps - it's a lofty ambition, but a worthy one.

The focus of the past 18 months has been to listen to the community and to stabilise the project. I think that goal isn't yet complete, but is close. There are libraries to upgrade, tests to write, documentation to add, and a general need to grow the community to support those steps. There is no way that 1 person can do this by themselves, as I have found out the hard way this year.

Eventually, I would like to get to the goals of 0.4 by way of refactoring 0.3, once tests and documentation are in place. I don't know how long it will take, but my company (Anephenix) is going to commercially sponsor work on SocketStream to get this done. I'm currently working for a major client in London, so my time is extremely limited.

I still have a lot of passion for SocketStream, and I want to see it succeed, in fact it's one of the reasons why Anephenix exists.

With regards to r.js, I do find that I prefer Browserify's approach - RequireJS' approach at times feels rather weird. That said, the asset pipeline of SocketStream has room for improvement. There is a need to make it handle unknown file formats better (as was the case with .map files), and to make it work better with bower components/ client-side node_modules.

[thepian]

This is now fixed in 0.4, which we hope to release in a month or so. If you want to try it out check out the next branch.