Programming - Implementing a Blockchain using Python and Flask [Dev Blog 8]

[Image 1]

Introduction

Hey it's a me again drifter1!

There's a lot to cover today, as I've been quite busy with changes. Minor changes include tweaking the ReadMe file / repository description to include a small installation guide and making the endpoints easier to read. But, there are also extremes like renaming the client to full node, and creating a new testing script that makes general requests to that full node.

So, let's first talk about why I changed the name of the client. Well, the functionality that we've been implementing to this point has been exactly the functionality that a full node in a blockchain system has. A full node validates transactions by keeping a complete copy of the blockchain. Of course, miners (block formers) and users (transaction posters) contact such nodes in order to get the necessary information. Thus, the blockchain network will consist of full nodes communicating with each other. This all will become clearer and clearer, as we get closer to the complete blockchain system.

Now for the various endpoint changes. First of all, in order to prevent import loops, which are quite annoying, the actual Flask endpoints and local and general requests helper functions have been split to separate files. Inside of these new "_requests.py" scripts we might later include the required functions to send transactions, block and utxo updates to all known nodes, but we shall see. Here might also be a good place to finally stop putting all the code inside of the same common source directory. Anyway, transactions can now only be posted when they are valid, as the callback for creating transactions now checks if each input is a valid UTXO output. Blocks are also checked more seriously by not only recalculating the hash, but also checking if all transactions are in the transactions.json (unconfirmed transactions) structure, and if all transaction inputs are valid UTXO entries and not referenced more than once (prevent double-spending). After the blocks are created, the included transactions are now removed from the unconfirmed transactions structure. And let's also not forget to mention that the UTXO's are also updated, as spent UTXOs (inputs) are removed and new entries are created for the outputs. Oh, and the balance is now also correctly calculated/updated whenever a UTXO output entry is added or removed!

The testing script now only post transactions and blocks, and the UTXO will be handled internally. We will get into an example run by the end of the dev blog...

So, without further ado, let's get more in-depth!


GitHub Repository


UTXO Balance Update

Let's start with the simplest change, as updating the balance of the UTXO was far easier than I first anticipated. Basically, in order to check if the output is in a block, we've already retrieved the block transaction output that was referenced by the utxo output. Well, now we also return it as a second return value, so that the utxo endpoint callbacks can use the output information.

The check utxo output in block function is as follows:

def check_utxo_output_in_block(settings: Full_Node_Settings, address: str, json_utxo_output: dict):
    utxo_output = json_destruct_utxo_output(json_utxo_output)
json_transaction_output = local_retrieve_block_transaction_output( settings, utxo_output.block_height, utxo_output.transaction_index, utxo_output.output_index)
if json_transaction_output == {}: return False, None
transaction_output = json_destruct_output(json_transaction_output)
if transaction_output.address == address: return True, transaction_output

So, updating the balance is as simple as:

json_utxo["balance"] += transaction_output.value
when adding a new utxo output, and:
json_utxo["balance"] -= transaction_output.value
when removing a utxo output.


Transaction Validation

In order to check if a transaction input is truly in the UTXO, another function has been defined, which makes a request to the corresponding UTXO endpoint. The check_transaction_input_in_utxo() function basically retrieves the UTXO output from the address and transaction hash pair defined in the input. And returns True if the output truly exists.

Thus, checking the inputs is a simple loop of calling this helper function:

json_transaction_inputs = json_transaction["inputs"]
for json_transaction_input in json_transaction_inputs: if not check_transaction_input_in_utxo(settings, json_transaction_input): # don't post transaction return {}

After this check all the other checks of course have to still be performed, like calculating the total_input and total_output, recalculating and checking the transaction hash, and verifying the signature of all inputs.


Block Validation

The biggest change is in the block creation callback...

After checking the hash by recalculating it, all transactions (except the coinbase/reward transaction) are checked for the following:

  • If they are in the unconfirmed transacitons array
  • If the inputs are in the UTXO
  • If UTXOs are referenced twice (prevent double-spending)

In code this is quite easy to implement:

json_transactions = json_block["transactions"]
json_unconfirmed_transactions = local_retrieve_transactions(settings)
for json_transaction in json_transactions:
# don't check coinbase transaction if json_transaction["inputs"][0]["output_address"] == "0x0000000000000000000000000000000000000000": continue
# check if transaction in unconfirmed transactions array if json_transaction not in json_unconfirmed_transactions: return {}
json_inputs = json_transaction["inputs"]
# keep track of referenced outputs json_checked_inputs = []
# check if inputs are in utxo for json_input in json_inputs:
if not check_transaction_input_in_utxo(settings, json_input): return {}
if json_input in json_checked_inputs: return {}
json_checked_inputs.append(json_input)

The coinbase transaction check might change later on...

So, the block passes all these checks, what now?

Well, first the block is created by creating the corresponding file. Then, all transactions except the coinbase transaction are removed from the unconfirmed transactions structure as follows:

for json_transaction in json_block["transactions"]:
# skip coinbase transaction if json_transaction["inputs"][0]["output_address"] == "0x0000000000000000000000000000000000000000": continue
local_remove_transaction(settings, json_transaction)
Yes, it's only a remove transaction endpoint request for each transaction.

And after that, the transaction inputs and outputs are used in order to update the UTXO's. For the coinbase transaction, a utxo output is created for the output only, so only cover the code for the remainder transactions. Well, in order to remove the inputs from the UTXO structure, the corresponding UTXO output is first retrieved using the address and transaction hash pair, and afterwards removed calling the remove utxo output:

for json_input in json_transaction["inputs"]:
    json_utxo_output = local_retrieve_utxo_output_from_address_and_transaction_hash(
        settings, json_input["output_address"], json_input["transaction_hash"])
local_remove_utxo_output( settings, json_input["output_address"], json_utxo_output)
For the output, a new UTXO output is created and added using the corresponding endpoint:
for json_output in json_transaction["outputs"]:
    utxo_output = UTXO_Output(
        block_height=json_block["height"],
        transaction_hash=json_transaction["hash"],
        transaction_index=transaction_index,
        output_index=json_output["index"]
    )
json_utxo_output = json_construct_utxo_output(utxo_output)
local_add_utxo_output( settings, json_output["address"], json_utxo_output)
Let's note that it's necessary to keep track of the transaction_index, as it is not stored as an JSON entry.
transaction_index = 0
for json_transaction in json_block["transactions"]:
...
transaction_index += 1


Testing Script Example Run

The testing script is basically the exact same as in the previous Dev Blog. But, because the UTXO's are now handled by the block creation callback, such requests are no longer necessary. Two more UTXO files will also be generated for the addresses referenced as outputs in the transaction.

The testing script posts the blocks and transactions to the first full node that it retrieves from the dns server. It first creates and posts a block without any transactions, except the coinbase transaction, which references to the wallet address of the testing client:

# retrieve first known node from dns server
json_node = general_retrieve_nodes(settings.main_dns_server_json_node)[0]
# load wallet information wallet_json = json.load(open(settings.wallet_path, "r")) address = json_retrieve_address(wallet_json) private_key = json_retrieve_private_key(wallet_json)
# create reward transaction reward_input = Input( output_value=2.5 )
reward_output = Output( address=address, value=2.5 ) calculate_output_hash(reward_output)
reward_transaction = Transaction( inputs=[reward_input], outputs=[reward_output], total_input=2.5, total_output=2.5, fee=0 ) calculate_transaction_hash(reward_transaction)
# create block 0 block0 = Block(height=0, creator=address, reward=2.5, fees=0, nonce="abcdef", transactions=[reward_transaction]) calculate_block_hash(block0) json_block0 = json_construct_block(block0)
general_create_block(json_node, json_block0)

This leads to the following UTXO file:

{
    "utxo_outputs": [
        {
            "block_height": 0,
            "transaction_hash": "8d0127a10de13ef97f61d4b5a72934e769d483654217d58ce94106703554b7cf",
            "transaction_index": 0,
            "output_index": 0
        }
    ],
    "balance": 2.5,
    "last_block_included": 0
}

Using this reward as an input, it then creates and post a transaction:

input0 = Input(
    transaction_hash=reward_transaction.hash,
    output_index=0,
    output_address=address,
    output_value=reward_output.value,
    output_hash=reward_output.hash,
)
sign_input(input0, private_key)
output0 = Output( index=0, address="0x60a192daca0804e113d6e6d41852c611be5de0bf", value=1.2 ) calculate_output_hash(output0)
output1 = Output( index=1, address="0xe23fc383c7007002ef08be610cef95a86ce44e26", value=0.3 ) calculate_output_hash(output1)
output2 = Output( index=2, address=address, value=0.999 ) calculate_output_hash(output2)
transaction = Transaction( inputs=[input0], outputs=[output0, output1, output2], total_input=2.5, total_output=2.499, fee=0.001 ) calculate_transaction_hash(transaction) json_transaction = json_construct_transaction(transaction)
general_post_transaction(json_node, json_transaction)

Of course this transaction is valid! So, after creating another reward transaction, another block can be created and posted:

block1 = Block(height=1, creator=address, reward=2.5, fees=0.001,
            nonce="123456", transactions=[reward_transaction2, transaction])
calculate_block_hash(block1)
json_block1 = json_construct_block(block1)
general_create_block(json_node, json_block1)

Retrieving the UTXO for the wallet address is as simple as:

json_utxo = general_retrieve_utxo_address(json_node, address)
which yields the following final UTXO:
{
    "utxo_outputs": [
        {
            "block_height": 1,
            "transaction_hash": "4a8d603467db70f8bd987b1bf64d8b6a487c21ebe4e839865f0bba1e00acfd27",
            "transaction_index": 0,
            "output_index": 0
        },
        {
            "block_height": 1,
            "transaction_hash": "02d52bc16514e24e60e22b92b14827c74a5ce222002fdc812768024595078cbd",
            "transaction_index": 1,
            "output_index": 2
        }
    ],
    "balance": 3.499,
    "last_block_included": 1
}

Just for reference, the final full_node directory looks like this:

In order to try this out, run a dns_server, a full_node and afterwards the testing.py script! :)


RESOURCES:

References

  1. https://www.python.org/
  2. https://flask.palletsprojects.com/en/2.0.x/
  3. https://docs.python-requests.org/
  4. https://pypi.org/project/mnemonic/0.20/
  5. https://pypi.org/project/ecdsa/
  6. https://docs.microsoft.com/en-us/windows/wsl/install-win10
  7. https://www.docker.com/
  8. https://code.visualstudio.com/
  9. https://insomnia.rest/

Images

  1. https://pixabay.com/illustrations/blockchain-block-chain-technology-3019121/
The rest is screenshots or made using draw.io

Previous dev blogs of the series


Final words | Next up

And this is actually it for today's post!

I will keep you posted on my journey! Expect such articles to come out weekly, or even twice a week, until I'm finished!

Keep on drifting!

H2
H3
H4
3 columns
2 columns
1 column
Join the conversation now