Feature Request: NameCoin Support

[PowerPress]

Please add NameCoin support. We really need a censorship resistant DNS solution that NameCoin provides along with many other great features such as identity.

https://wiki.namecoin.info/?title=Identity

But there is no way to do cold storage for NameCoin and this really is Armory's area of expertise. I beseech you to add NameCoin integration for this.

[JeremyRand]

Hi,

I'm a Namecoin developer. I'd really like to see this as well, and I might be able to help with coding it depending on what's involved (I haven't worked with the Armory source code yet). If someone is able to get this coded (let's say with a command line flag to switch to Namecoin mode), would the Armory team be willing to merge it?

Cheers.

[int03h]

That would be neat !! +1
On Aug 13, 2014 12:04 AM, "JeremyRand" notifications@github.com wrote:

Hi,

I'm a Namecoin developer. I'd really like to see this as well, and I might
be able to help with coding it depending on what's involved (I haven't
worked with the Armory source code yet). If someone is able to get this
coded (let's say with a command line flag to switch to Namecoin mode),
would the Armory team be willing to merge it?

Cheers.

—
Reply to this email directly or view it on GitHub
#230 (comment)
.

[josephbisch]

Hi @JeremyRand!

I am working on Namecoin integration in Armory. I would love it if you or someone from the Namecoin team could help me out. I got all the Python code done (except for replacing instances of the text Bitcoin or BTC with Namecoin or NMC.

I'm running into issues on the C++ side. You can see the bottom of this thread to see the latest place I am stuck. Right now we are wondering if the blk0001.dat file used by Namecoin might be a different format than that used by Bitcoin (maybe because of the special types of transactions Namecoin has)? I am stuck at the stage where Armory builds it's db from the block files, but I just get block parsing errors.

Thank you in advance.

Cheers,
Joseph

[JeremyRand]

Hello @josephbisch ,

That's awesome that you're working on this. Name transactions in Namecoin are actually valid (though nonstandard) Bitcoin transactions, so that shouldn't be the problem. There are two differences between Namecoin and Bitcoin's blockchain that I can think of here. First, Namecoin is based on Bitcoin 0.3.x, which used BDB rather than LevelDB as current Bitcoin releases do. It sounds from the forum thread that you're already aware of this. The other one is that Namecoin blocks can have a merged mining header. This sounds more likely to be what you're encountering. Merged mining started at block 19200 (although I believe the protocol allows blocks after that point to not have a merged mining header; it's optional, but almost all blocks after that point will probably have the header because it's more profitable for miners), so that would be consistent with the fact that there's an initial part of the blockchain that doesn't error for you. I'm not particularly familiar with the merged mining header's structure as it relates to the blockchain; @domob1812 is our main C++ guy and he will probably be able to help more than I can.

Thanks so much for working on this.

Cheers,
-Jeremy

[JeremyRand]

As an aside, @domob1812 is working on porting Namecoin to the latest Bitcoin Core codebase; once that's finished, it will use LevelDB like Bitcoin Core (but it will still have the merged mining header). So, you might want to make sure that you can adjust your Namecoin integration code to use a LevelDB blockchain, like Bitcoin, when that happens. No ETA exists for that work yet as far as I know.

[domob1812]

I also agree that Namecoin support for Armory would be great! Maybe it could, at some point in the future, even become the default UI (I'm currently not planning to immediately update the Qt UI with the mentioned rebase - that would be an opportunity to switch to a separate UI based off the daemon as core). So I also offer to help where I can.

The rebase onto Bitcoin Core is progressing well, so I also think you shouldn't invest too much time into implementing specific solutions for the old BDB codebase (in case that's necessary for you). @JeremyRand is right that the initial part of the blockchain will be accepted as-is by Bitcoin code, which I also observed when starting the rebase.

I've recently read through the merge-mining code, and now believe that I have a thorough understanding of the involved data structure and format - let me know if I can help you out on that. The rebased version will also feature multisig support (after an appropriate waiting time and hardfork), which, I think, fits well to Armory.

[josephbisch]

It's good that the rebase is progressing well. There is no need to implement BDB specific code (because the underlying format on disk is the same). But I did need to change the filenames and locations for the block dat files and will need to disable multisig and P2SH support for now (though it shouldn't be too hard, that should all be done in Python).

If you can explain the difference in the data structure, that would be helpful. I could look through the code myself, but I assume it would be quicker for you to explain it. Something like this, or just a sentence explaining where the merged mining header is.

[domob1812]

I think what you are interested in is CAuxPow's serialisation format. It is based on the format for CMerkleTx plus some extra data, and can be found in Namecoin's auxpow.h. CBlock also contains an extra field auxpow (with this format), as you can see in CBlock's serialisation (in main.h). I hope that helps - I don't think there's much "formal spec" around.

[josephbisch]

It looks to me from these lines that Namecoin still starts with the version and then the hash of the previous block. You can see for example that this block starts with the version, but then the next 32 bytes don't have any leading zeros, so I don't think they are a valid hash of a block. Maybe I am reading the Namecoin code wrong and it's not the prevBlockHash that follows the version. Which is weird because it looks to me like the auxpow goes after the nonce and before the transactions.

I downloaded the blockchain twice, so I know the dat files aren't corrupt.

[domob1812]

Yes, the format should be the same as Bitcoin's, except possibly adding an auxpow. The prevBlockHash need not necessarily start with zeros, since merge-mined blocks don't do that in general.

[JeremyRand]

Right, a merge-mined block will generally not have a hash with leading zeros.

[josephbisch]

I gave up on the Namecoin support in Armory, because I am not good with C++. I made a final post with the diff of what I have so far. If anyone wants to help with the C++ side of things, then I could help with whatever is left on the Python side. Thanks for the help, sorry I couldn't finish the Namecoin support.

[JeremyRand]

@josephbisch Any chance you could upload your current code to a GitHub repo (forked from Armory's repo)? That would make it somewhat easier to work with.

[josephbisch]

Sure thing, will do.

[josephbisch]

See my namecoin branch (based off of the dev branch, like Armory Technologies, Inc wants).

[josephbisch]

I tried parsing the blk0001.dat file in Python to get a better understanding of the format and I made a breakthrough. Since blocks >= 19200 may or may not be merge mined, I was wondering how to determine whether or not a particular block is merge mined. If a block is merge mined it's version in the block header is 0x10101 and if it is a regular block it has version 0x1.

I had been trying to check to see if the block height was >= 19200 or not, which doesn't work, because you can't know the block height when you are in the middle of parsing the block files. Plus as I said blocks >= 19200 aren't necessarily merge mined.

I'll take another look at the C++ code tomorrow. I might be able to figure it out now that I have this figured out.

[JeremyRand]

Ah, nice work. Good luck with the C++ tomorrow; let us know if you have any trouble and we'll try to help.

[domob1812]

Yes, that's true. Actually, version 0x01 is only for "legacy" blocks. After 19,200 (and actually starting a bit earlier), blocks are either 0x010001 (ordinary, e. g., 19,204) or 0x010101 (merge-mined, e. g., 19,200). The 0x010000 bit is the merge-mining chain ID (Namecoin = 1, other coins have other values) and is present always. To determine whether or not a block is merge-mined, look at the 0x0100 bit (BLOCK_VERSION_AUXPOW in main.h).

[josephbisch]

I (think) I got the all the blocks loaded without errors for the Namecoin testnet (I haven't tried with mainnet yet). Now I need some testnet namecoins to verify that Armory displays the balance correctly.

I tried using a faucet a while ago, but I don't think it worked, because my balance is still 0. I also tried to start Namecoin-qt with the -gen flag set, but it didn't seem to do anything. If someone can send me some testnet namecoins, that would be very helpful!

My address: mhYDTcGBqmkNjt8kaZDsexC2Bmb8yZfK7B.

[josephbisch]

Wow, someone sent me 50 testnet namecoins! (Yes I know they are worthless, still surprised at the number)

The balance does show up. I need to do some more testing, with sending and receiving. I have a feeling that the fee calculation logic will need to be changed. Also P2SH and multisig will need to be disabled for now.

[domob1812]

That was me. :) I still have tons of them after I mined a bit before there was regular mining going on and the difficulty got so insanely high. Good luck with your work!

[josephbisch]

Thanks, I was able to send successfully too.

[josephbisch]

I pushed my latest changes to my fork of Armory. It can't be tested right now because the dev branch which I based the namecoin branch off of is broken. I'll post another comment here when it is working again, so that people who want to can test Namecoin support in Armory.

My next step is to change the fee calculation logic, because Namecoin appears to have different fee calculation logic. It looks like one of the more difficult parts of Namecoin integration. The file armoryengine/CoinSelection.py has, at the bottom, three functions for calculating fees. The multisig function can be skipped, since Namecoin doesn't support multisig. So that leaves calcMinSuggestedFees and calcMinSuggestedFeesNew that need to be modified for Namecoin.

I see main.h has a GetMinFee function. So I just need to translate that logic to Python.

After the fees, I think Namecoin support in ArmoryQt will be complete. The C++ still has Namecoin hardcoded, so that will need to be changed and a unit test for Namecoin will need to be written. Then armoryd will need work.

[josephbisch]

The dev branch is working again and I pushed my changes again, so you can test it now if you want. I'm not done with the fee calculation logic yet.

Let me know if you encounter any errors. I recommend you just test with testnet coins. If you have any issues, disable the option in Armory settings to have Armory manage the daemon for you and try manually starting namecoind.

To test:
clone my repo
switch to the namecoin branch
run make
run python ArmoryQt.py --namecoin-testnet

[josephbisch]

I tried setting the same rules for calculating fees in calcMinSuggestedFeesNew as I did in calcMinSuggestedFees, but Armory is allowing me to attempt to send a single satoshi with no fee (then I obviously get the popup about the transaction not being accepted by the network).

Could someone who is familiar with the Namecoin transaction fee calculation rules look at my commit where I try to add the logic to Armory?

[josephbisch]

I pushed some code for the fee calculations. In my tests, transactions less than 0.01NMC calculates a 0.005NMC fee and anything greater than or equal to 0.01NMC calculates no fee. I need to figure out how to create larger sized transactions for testing purposes to see if the fee scales with transaction size as it should.

[JeremyRand]

@josephbisch I assume that if you send coins to a bunch of addresses in 1 tx, that would be sufficient to get above the 1 KB level.

I vaguely recall something about Namecoin-Qt's default fees being raised a few months ago (I think 0.005 NMC minimum on all transactions by default), but I might be wrong... maybe @domob1812 remembers more about that than I do?

[JeremyRand]

Maybe this is the commit I'm thinking of? namecoin/namecoin-legacy@b23e7dc

[josephbisch]

I don't think that commit matters for Armory. That looks like just the default fee that the Namecoin-qt client uses and that can be changed in the settings. What does matter is MIN_TX_FEE and MIN_RELAY_TX_FEE and how the fee is calculated using those.

[JeremyRand]

Yes, I think you're right on that.

[josephbisch]

And it looks to me like I need to get above 3K, because the block itself is 1K. See this line. Unless I am looking in the wrong place.

[josephbisch]

I pushed an update to the fee calculations, so that 0.005NMC is charged per output less than 0.01NMC. It is these lines in the Namecoin source code.

I tried outputting to multiple addresses, but I am actually having a hard time getting a fee with the exception of the less than 0.01 output fee. I might have to generate a transaction to 100s of addresses programmatically, because it doesn't look feasable to create a large enough transcation by hand with the GUI. Maybe this is a sign that I did something incorrect with the fee calculation code.