v0.7.0b2

This is the release notes of v0.7.0b2. See here for the complete list of solved issues and merged PRs.

Changes that break compatibility

• From v0.7.0b2 on, staled threading-based scheduler as well as distributed scheduler based on Mars actor 1.0 have been removed, thus clients with older versions are completely incompatible.

Highlights

• Unified scheduling based on Oscar which is Mars actor 2.0 is ready for tests.

New Features

• DataFrame
  • Add add_prefix support (#2132, thanks @aeinrw!)
• Services
  • Services web handler and api (#2102)
  • Implements lifecycle service (#2117)
  • Add initial implementation of scheduling service (#2111)
• Deloy
  • Add command line support for Oscar deployment (#2131)
• Ray
  • [Ray] ray oscar deploy (#2089)

Enhancements

• Hold data ref in DataManager (#2090)
• Enabling iterative tiling etc support for task service (#2097)
• Enable optimization for task service (#2098)
• Implement last_idle_time API (#2099)
• Integrate mars object check for session (#2103)
• Make Mars pools compatible with Python 3.6 (#2110)
• [ray] optimize ray deploy speed (#2118)
• Implement RESTful web API (#2120)
• [ray] Support supervisor exclusive node option (#2121)
• Add asyncio task timeout debugger (#2127)
• Add transfer support for storage service (#2100)
• [ray] supervisor support sub pool (#2128)
• Configure azure pipelines job timeout (#2139)
• Allow overriding service config files (#2140)

Bug fixes

• Fix distributed make_blobs and column pruning in read_sql (#2092)
• Fix result error when yield after exceptions (#2096)
• Wrap sync method in session so that they will be running in threads (#2109)
• Fix pool cases and shared memory cases in Windows (#2114)
• Fix _get_ports_from_netstat hang (#2116)
• Fix unpickle mars config error (#2130)
• Filter pipeline jobs by branch (#2138)

Tests

• Fix coverage result on SubActorPool (#2095)
• [Ray] Support ray subprocess covarage (#2101)
• Run operand tests in Azure Pipelines (#2137)
• Migrate tensor/dataframe/learn tests to oscar (#2106)