Yep. Composer is dependency management so dev stuff. It replaces committing a copy of library into your plugin or adding it as subrepository with listing it as dependency in its composer.json configuration file.
So in plugin I work on right now I have something like this in its composer.json (I set up stub definition for TLC Transients in my custom Composer repo http://www.rarst.net/packages.json ):
So when I install my plugin with Composer copy of TLC Transients is automagically downloaded (or checked out from GitHub depending on command, settings and environment) and made available to plugin.
I am still exploring Composer with WordPress and it's not all smooth (mostly due to WP being legacy code base from the view point of PHP 5.3+ world), however it's fantastic tool and is quickly becoming de-facto standard for sharing and reusing PHP libraries.