New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bitcoin-Intelligent-Life - Blockchain engineering class project #4039

Open
synctext opened this Issue Nov 19, 2018 · 14 comments

Comments

@synctext
Copy link
Member

synctext commented Nov 19, 2018

Project : Bitcoin-Intelligent-Life

Self-replicating Bitcoin-based entities using deep re-enforcement learning

You will create a key prototype that advances the state-of-the-art of self-replicating software, autonomy, and artificial intelligence. Your mission is not to terminate all human life.

image

Bitcoin is an essential technology for digital life: you can buy servers. In prior work TUDelft has created CloudOmate. With CloudOmate you can buy servers with Bitcoin automatically. CloudOmate gives any computer the power to own money, replicate freely, no human can access that server, and no human can take that money away. It becomes an autonomous virtual entity. With cloudomate can even enhance it's own privacy and buy VPN networking autonomously.

Next step is adding intelligence and learning to this autonomous virtual entity. You will combine buying servers with Bitcoins with the another peak-hype technology of today: AI. You will create a basic intelligent entity, capable of self-replication and learning capability. For the learning side you can use deep re-enforcement learning engine in Python. See basic tutorial

Possible sprints for the 10 weeks:

  • get CloudOmate going and repair broken APIs to VPS providers and payment gateways
  • create the first primitive Q-table in Python with the focus on survival
  • add primitive support for decentral market trading and income earning
  • focus on source of sustainable income for the use existing Python code for Tor-like exit node services

Outcome: running code and Github readme.md (no thick final report that nobody really reads).

Warning: this is a scientifically most challenging assignment (recommended for 'Cum Laude' level students)
Supervised by: Dr. Johan Pouwelse

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Nov 19, 2018

Kick-off meeting:

@devos50

This comment has been minimized.

Copy link
Contributor

devos50 commented Nov 19, 2018

Please note that the 2017 plebnet project has been continued by another group this year. See the last few posts of this ticket: #2925

Report: https://repository.tudelft.nl/islandora/object/uuid%3A614efc19-e6a9-453f-8084-2e2c54ddd93f

They made this visualization:
42031541-16e66e28-7ac6-11e8-8ddd-c2db87de599f

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Nov 27, 2018

Meeting minutes:

  • time spend mostly coding, less about reading prior stuff
  • MattSkala/cloudomate#7
  • documented in a single list an overview of prior work (from 2016 onwards)
    • create single table overview of prior groups
    • github, final report, issue, and cumulative student numbers
    • (no need for student names if you have GithubIDs)
  • find other providers, using the cart.php trick
  • plebnet self-replication, docker needs repairing or something is broken

Next sprint ideas:

  • focus on Plebnet operational
  • wrap up cloudomate
  • first Q-table reward function and global agent design
    • deliberately as primitive as possible
    • select between VPS providers
      • state: current VPS provider
      • action: stay or switch
    • market strategy
      • blind and static
      • do not take into account you inventory, demand or pricing fluctiations (next sprints)
      • earn Tribler tokens for whole month and sell everything in last 24h of VPS account
      • 1 offer on the market, price: sell everything at price point to buy exactly 1 new VPS
    • Tor-like Tribler exit traffic only, no seeding or credit mining yet
    • upon death: global recovery wallet

@qstokkink qstokkink added this to the Backlog milestone Nov 27, 2018

@andrebreis

This comment has been minimized.

Copy link
Contributor

andrebreis commented Dec 2, 2018

@Bloodyfool

This comment has been minimized.

Copy link

Bloodyfool commented Dec 2, 2018

Timeline of the PlebNet and Cloudomate Projects

This post serves to give an overview of the history of the PlebNet and Cloudomate projects. It aims to give a high level overview of the project groups that have worked on the project and their contributions. 5 teams have worked on this project before we (team 6) got started.

Team 1: TENNET (Skynet 2.0)

This is the inception of the project. The project started in april 2016 as the bachelor end project of:

In this first project a miminum working project was created. Skynet 2.0 added:

  • primitive automation of buying VPS's
  • primitive self replication to other machines using this automation given
    enough bitcoin.
  • A primitive DNA structure for genetic learning

After this project there was not working version yet as money making was not properly implemented.

Documentation:

Team 2: PlebNet

The PlebNet project started in May 2017 as the bachelor end project of:

This project improved on Skynet 2.0 by:

  • extracting the VPS buying code into a separate project called cloudomate, with restructured code, better web scraping techniques, and more VPS providers implemented.
  • expanding on the DNA structure and mutations.
  • implementing:
    • Tribler reputation mining
    • trading reputation for Bitcoin
    • buying servers with Bitcoin
    • installing itself on the servers to start mining reputation again

Documentation:

Team 3: Growing Life using Blockchain

Starting November 2017 the PlebNet project was improved upon by:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

  • Fixing broken VPS'
  • Migrating to python3
  • Adding automated VPN purchasing

The project improved upon PlebNet by:

  • Integrating it with their improved version of Cloudomate

Documentation:

Team 4: Growing primitive blockchain life

Starting November 2017 as well, the PlebNet Project was improved upon by:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

  • adding automated VPN purchasing
  • found a way around captchas and google ReCaptcha

The project improved upon PlebNet by:

  • adding the option to use an Etherium wallet

Documentation:

Team 5:

Starting April 2018 the PlebNet project was improved upon as the bachelor end
project of:

The project improved upon Cloudomate by:

  • fixing broken VPS'
  • adding a random user generator

The project improved upon PlebNet by:

  • dynamically checking what VPS providers cloudomate has access to at the
    moment
  • restructuring the program architecture
  • modeling and documenting the project
  • Making PlebNet communicate back over IRC

Documentation:

Team 6:

Starting November 2018 the Plebnet project is being improved by:

Documentation:

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Dec 2, 2018

Impressive overview! 25 developers...

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Dec 3, 2018

Meeting minutes. Current status: simple learning and reproduction operational. Created wealth of 20GBytes in 2 days and creating market orderbook entries.

Continuous procurement bot buys from the self-replicating bots. This is still under construction.
Next problem "derive efficient representations of the environment" for Q-tables. When each step in learning takes 1 month, you learn very slowly. Possible solution: robot learning gossip for parallel learning.

Upcoming sprint: agents will have a character class which determines: appetite for risk, long-term versus short term thinking, stubbornness versus flexible. Q-tables in general and CloudOmate in general. CI?

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Dec 10, 2018

Meeting minutes: Current status: initial Q-table code created, multiple market strategies, and partial fulfillment of orderbook entries.

Upcoming sprint goal: end-to-end test

Idea of collective robot learning seems viable. First mechanism is message exchange between robots, assuming full trust, complete honesty and no attackers. Use IPv8 tutorial to publish your selected public-key, announce using our DHT, and receive incoming messages. Remove IRC server based mechanism in Plebnet. Use messages in Protocol buffers serialisation. Example demo messages to implement: "full Q-table sharing", keep it simple and atomic. Only share after you learned something and "edited it".

To call it "deep reinforcement learning" we need a neural network and historical performance data, yield of market strategies, and hardware performance (intro). Future sprint could use a testnet and orderbook replay, see our prior work with limit orders. Do the bots need to crawl the IPv8 market community to build transaction history database of last 30days? Please avoid tensorflow or other gigantic libraries, keep it simple and maintainable for easy bot-install (like here).

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Dec 17, 2018

Meeting notes:

  • finishing prior sprint stuff; QTable, market strategies, VPS selection
  • operational: "Hello World" messages between bots. Towards collective learning.
  • IRC still there for easy debugging
  • initial code for market transaction history; adding Simple Moving Average
  • Two active exit node agents
  • Focus on "collective reinforcement learning" and actual deployment
  • Continuous procurement bot deployment ("buy bot")
  • Plebnet renaming? ('Dolly Bot'?)
@synctext

This comment has been minimized.

Copy link
Member

synctext commented Dec 18, 2018

Thoughts; please use as basis for your readme.md:

  • Darwinian reinforcement learning
  • "Internet-deployed Darwinian reinforcement learning based on self-replicating"
  • advancement of our own gossip learning done within TUDelft / Szeged collaborative project in 2011
  • breaks with tradition by not having a single trained model
  • scales because it does not attempt to create a single globally consistent model
  • continuously exchange training data; never merge into a single trained model
  • each node or agents has their own unique machine learning model
  • very uncomfortable: probabilistic learning depending on the individual model (Thnx Jan! for this insight)
  • "Darwinator" = Darwin + terminator = heretic scientist + killer robot :-)
@Harapuia

This comment has been minimized.

Copy link

Harapuia commented Dec 18, 2018

I'm still monitoring this issue by the by, If you have any questions or want to yell at me for what we did to cloudomate feel free to shoot a message.

@synctext

This comment has been minimized.

Copy link
Member

synctext commented Jan 8, 2019

Meeting minutes:

  • This week wrap up the features and test.
  • Future Sprints: end-to-end replication tests with increasing size (feature freeze/limited extensions only?)
  • Now implemented: deterministic QTable selection. Survival/Darwinian performance is enhanced with non-determinism in VPS self-replication. Fancy simulated annealing optimization or just an exponentially deceasing probability to select the No1. choice; No.2 VPS provider with much less probability, etc.
  • non-blind market strategies ongoing work
  • crashed Tribler wallet, untested code for auto-buy-bot.
  • TUN/TAP and is VPN really enabled?
@synctext

This comment has been minimized.

Copy link
Member

synctext commented Jan 14, 2019

Next week: end-to-end replication tests (and screen capture of cmdline demonstration on Youtube)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment