Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
125 lines (80 sloc) 5.08 KB
title layout tags permalink
TIL notes or Stuff I Learned Most Recently
post
til
python
pex
performance
perftests
gatling
faketdd
websockets
nginx
geoip
docker
bi
redash
postgresql
macosx
2018/03/aggregate-recent-tils

Once again I have started to write down TIL (Today I Learned) notes in my notebook & push some of that here, both as a way to share useful things and a reminder to myself. TIL sysyphus process Copyright 2011-2015 industrial-evolution from DeviantArt
Humble request - if you have discovered the page via google & found it useful - please leave a comment. That's it.

TILs

In random appearance order

Websockets

  • Chrome built-in WS (Web socket) traffic monitoring.

    I have most recently used Firefox as my main browser & development environment. Turns out - most recent Firefox has broken (e.g. Firefox is in transition to new engine parts phase) web socket traffic monitoring and respective plugin support. Not the case with Chrome. chrome ws example

Performance Tests

  • Gatling

    For most of my professional career, I have mostly used tools like Apache AB and Jmeter in generic performance testing. This time I needed WS (WebsScket) support and it made a compelling case to try something new.

    Pros:

    • DSL scriptable == flexible
    • Simulation setup is done in virtual user terms, not in traditional RPS (request per second) terms, which gives more high level and IMO more useful perspective.

    Cons:

    • DSL, in this case, is Scala
    • Mediocre documentation. Still could find everything that needed, but with extra effort.
  • flood.io superb performance test execution service.

    BTW featured in ThoughtWorks 2018 Technology Radar. Basically, you just upload your performance test scripts, select how many nodes you want to execute them, get aggregated execution report with no hassle.

    Not affiliated in any sense, happy customer, highly recommend.

TDD

After attending DevTernity 2017: Ian Cooper - TDD, Where Did It All Go Wrong session - we had a small epiphany to redesign most of our tests from Mockito mocks to FakeRepository.

Now the test maintenance process is fewer tears and more joy. Retrospectively, most of the tests were easier to adjust to using FakeRepository than "fix" them.

HTTP Middleware

  • ALB header rewrite is confusing.

    We were forced to use Forwarded header to make WebSockets work with Spring (Java) implementations.

  • Nginx GeoIP module.

    Initially, I developed GeoIP detection on top of http://freegeoip.net/ service.

    Later on, we started to use CloudFlare with CF built in GeoIP data. So, we had an option to remove hundreds of code lines, simplify and reduce latencies significantly.

    Then I have discovered Nginx GeoIP module as an exit strategy in places we do not plan to use CloudFlare.

    Things could not be better.

Business Intelligence & PostgreSQL

  • Redash

    Redash in action

    I have discovered Redash service and I love it. E.g. simple way to combine Database and Google Analytics data in a single business report. Awesome.

  • PostgreSQL conditional Aggregated queries

    See example below:

    SELECT
          count(*) AS unfiltered,
          count(*) FILTER (WHERE i < 5) AS filtered
      FROM generate_series(1,10) AS s(i);
      unfiltered | filtered
      ------------+----------
              10 |        4
      (1 row)
    

Random (MacOS X, docker,..)

  • Docker on Mac, performance degradation.

    Fix - add localhost to /etc/hosts. Refer to https://github.com/docker/compose/issues/3419 for more details.

  • MacOS X - autohide menu (status bar) + dock == eternal && fast fullscreen

    • General- Automatically hide and show menu bar
    • Dock - Automatically hide and show the Dock

Python

  • Use decode("utf-8") to achieve camel case with Russian str.

    RE_PUNCTUATION = re.compile(r'[\s{}]+'.format(re.escape(string.punctuation)))
    
    def namify(name):
      if isinstance(name, str):
          cap = string.capwords(name.decode('utf-8'))
      else:
          cap = string.capwords(name)
      return ''.join(RE_PUNCTUATION.split(cap))
  • PEX - "cross-platform-binary" format.

    We use PEX (Python EXecutable) as a way to bundle devops python scripts into one single .pex file.

    So most of the operations are as simple as scp deploy.pex .. && deploy.pex --arg1 --arg2,

Voilà

More to come.