Problem retrieving the content of zero value transactions from the testnet

Question

I am sending two zero value transactions to a specific address on the testnet. The content is shown below:

This is the address I am sending it to:
DZALQJKNDFBKADHCKKWZXXCGVCNHOIIARUPEJO9CILLTDDAZPEDZX9TMUWEBEPYFHFMHNZBXKZQO9NMZ9

declaration 1: 
{'name': 'Gepp', 'surname': 'Balestrieri', 'birthdate': '01/01/1900', 'id': 'YOLOXXX', 'time': '12:00', 'temperature': '36.5', 'reason': 'reason1', 'destinations': [[5.72782, 45.18831], [5.72781, 45.18832], [5.72778, 45.18837], [5.72777, 45.18838], [5.72768, 45.18835], [5.72763, 45.18838], [5.72755, 45.18836], [5.72749, 45.18834], [5.72671, 45.18807], [5.72665, 45.18805], [5.72653, 45.18802], [5.72643, 45.18799], [5.72634, 45.18796], [5.72571, 45.18792], [5.72506, 45.18788], [5.72484, 45.18787], [5.72427, 45.18785], [5.72414, 45.18784], [5.7239, 45.18782], [5.72366, 45.18781], [5.72354, 45.1878], [5.72335, 45.18779], [5.7233, 45.18779], [5.72306, 45.18781], [5.72256, 45.18784], [5.72181, 45.18788], [5.72164, 45.18789], [5.72146, 45.1879], [5.7207, 45.18794], [5.71932, 45.18803], [5.71925, 45.18803], [5.71924, 45.18799], [5.71923, 45.18795], [5.71921, 45.18792], [5.71911, 45.18792], [5.71903, 45.18793], [5.71899, 45.18793], [5.71891, 45.18793], [5.71879, 45.18793], [5.7188, 45.18798], [5.71793, 45.18804], [5.71786, 45.18805], [5.71782, 45.18805], [5.71777, 45.18805], [5.71776, 45.18805], [5.71678, 45.18811], [5.71674, 45.18805], [5.71654, 45.18772], [5.71655, 45.18769], [5.71663, 45.18752], [5.71659, 45.18746], [5.71642, 45.18749], [5.71642, 45.1875], [5.71642, 45.18753], [5.71648, 45.18752], [5.71653, 45.18754], [5.71656, 45.18757], [5.71657, 45.18761], [5.71656, 45.18762]]}
Declaration 2: 
{'name': 'Giovanni', 'surname': 'Lucente', 'birthdate': '01/01/1900', 'id': 'YYYYYYY', 'time': '12:00', 'temperature': '36.5', 'reason': 'reason1', 'destinations': [[5.72545, 45.18582], [5.72513, 45.18571], [5.72509, 45.18569], [5.725, 45.18566], [5.72489, 45.18563], [5.72323, 45.18509], [5.72318, 45.18507], [5.72314, 45.18506], [5.72312, 45.18509], [5.7229, 45.18539], [5.72246, 45.18523], [5.72242, 45.18529], [5.72238, 45.18535], [5.72236, 45.18536], [5.72236, 45.1854], [5.72228, 45.18544], [5.72226, 45.18546], [5.72225, 45.18547], [5.72218, 45.18546], [5.72214, 45.18546], [5.72201, 45.18547], [5.72139, 45.18555], [5.72074, 45.18563], [5.72029, 45.18568], [5.72019, 45.18569], [5.72014, 45.1857], [5.72018, 45.18576], [5.72014, 45.18578], [5.7188, 45.18662], [5.71875, 45.18665], [5.71879, 45.18676], [5.71872, 45.18678], [5.71865, 45.18679], [5.71861, 45.1868], [5.71854, 45.18681], [5.71846, 45.18682], [5.71855, 45.18708], [5.71849, 45.1871], [5.71747, 45.18741], [5.71744, 45.18742], [5.7174, 45.18743], [5.71705, 45.18754], [5.7167, 45.18764], [5.71663, 45.18752], [5.71659, 45.18746], [5.71642, 45.18749], [5.71642, 45.1875], [5.71642, 45.18753], [5.71648, 45.18752], [5.71653, 45.18754], [5.71656, 45.18757], [5.71657, 45.18761], [5.71656, 45.18762]]}

I would like to retrieve the content of those transactions. I am using the following python code for this purpose:

from iota.commands.extended import find_transaction_objects
from iota import Iota, Address

# Declare an API object
api = Iota('https://nodes.devnet.iota.org:443', testnet=True)

add = input('Insert Address from last milestone: ')

list_add = [Address(add)]
response = api.find_transaction_objects(addresses=list_add)

if not response['transactions']:
    print('Couldn\'t find data for the given address.')
else:
    print('Found:')
    # Iterate over the fetched transaction objects
    for tx in response['transactions']:
        # data is in the signature_message_fragment attribute as trytes, we need
        # to decode it into a unicode string
        data = tx.signature_message_fragment.decode(errors='ignore')

        print("\n\n"+str(data))

And the result is not was I was expecting:

Insert Address from last milestone: DZALQJKNDFBKADHCKKWZXXCGVCNHOIIARUPEJO9CILLTDDAZPEDZX9TMUWEBEPYFHFMHNZBXKZQO9NMZ9
Found:
{"name": "Gepp", "surname": "Balestrieri", "birthdate": "01/01/1900", "id": "YOLOXXX", "time": "12:00", "temperature": "36.5", "reason": "reason1", "destinations": [[5.72782, 45.18831], [5.72781, 45.18832], [5.72778, 45.18837], [5.72777, 45.18838], [5.72768, 45.18835], [5.72763, 45.18838], [5.72755, 45.18836], [5.72749, 45.18834], [5.72671, 45.18807], [5.72665, 45.18805], [5.72653, 45.18802], [5.72643, 45.18799], [5.72634, 45.18796], [5.72571, 45.18792], [5.72506, 45.18788], [5.72484, 45.18787], [5.72427, 45.18785], [5.72414, 45.18784], [5.7239, 45.18782], [5.72366, 45.18781], [5.72354, 45.1878], [5.72335, 45.18779], [5.7233, 45.18779], [5.72306, 45.18781], [5.72256, 45.18784], [5.72181, 45.18788], [5.72164, 45.18789], [5.72146, 45.1879], [5.7207, 45.18794], [5.71932, 45.18803], [5.71925, 45.18803], [5.71924, 45.18799], [5.71923, 45.18795], [5.71921, 45.18792], [5.71911, 45.18792], [5.71903, 45.18793], [5.71899, 45.18793], [5.71891, 45.18793], [5.71879, 45.18793], [5.7188, 45.18798], [5.71793, 45.18804], [5.71786, 45.18805], [5.71782, 45.18805], [5.71777, 45.18805], [5.71776, 
788787877S7R77R77777777
7R77777777
{"name": "Giovanni", "surname": "Lucente", "birthdate": "01/01/1900", "id": "YYYYYYY", "time": "12:00", "temperature": "36.5", "reason": "reason1", "destinations": [[5.72545, 45.18582], [5.72513, 45.18571], [5.72509, 45.18569], [5.725, 45.18566], [5.72489, 45.18563], [5.72323, 45.18509], [5.72318, 45.18507], [5.72314, 45.18506], [5.72312, 45.18509], [5.7229, 45.18539], [5.72246, 45.18523], [5.72242, 45.18529], [5.72238, 45.18535], [5.72236, 45.18536], [5.72236, 45.1854], [5.72228, 45.18544], [5.72226, 45.18546], [5.72225, 45.18547], [5.72218, 45.18546], [5.72214, 45.18546], [5.72201, 45.18547], [5.72139, 45.18555], [5.72074, 45.18563], [5.72029, 45.18568], [5.72019, 45.18569], [5.72014, 45.1857], [5.72018, 45.18576], [5.72014, 45.18578], [5.7188, 45.18662], [5.71875, 45.18665], [5.71879, 45.18676], [5.71872, 45.18678], [5.71865, 45.18679], [5.71861, 45.1868], [5.71854, 45.18681], [5.71846, 45.18682], [5.71855, 45.18708], [5.71849, 45.1871], [5.71747, 45.18741], [5.71744, 45.18742], [5.7174, 45.18743], [5.71705, 45.18754], [5.7167, 45.18764], [5.71663, 45.18752], [5.71659, 45.1

The content seems to be compromised. Is the length of the message the cause of this problem? How can I solve this problem?

SOLUTION

Here is the new python code!

from iota.commands.extended import find_transaction_objects
from iota import Iota, Address
from pprint import pprint

# Declare an API object
api = Iota('https://nodes.devnet.iota.org:443', testnet=True)

add = input('Insert Address from last milestone: ')

list_add = [Address(add)]
response = api.find_transaction_objects(addresses=list_add)


list_bundle_hash = []

if not response['transactions']:
    print('Couldn\'t find data for the given address.')
else:
    print('Found:')
    # Iterate over the fetched transaction objects
    for tx in response['transactions']:
        # If it is a tail, save the hash to a list
        if tx.is_tail:
            print(tx.hash)
            list_bundle_hash.append(tx.hash)

for i in list_bundle_hash:
    # Fetch the corresponding bundle objects
    try:
        bundle = api.get_bundles([i])['bundles'][0]
        content =   bundle.get_messages()
        print(content)
    except Exception as e:
        print("Problem here")
        pprint(getattr(e, 'context', {}))

Answer 1

The problem is caused by the too long message. When you send data with the IOTA Python library (PyOTA) to the Tangle, it first converts the unicode message into trytes, then puts these trytes into the signature/message field of a transaction.

When the converted trytes are too long to fit in one transaction, it splits them across two transactions. Now comes the fun part for your case:

• The converted trytes are always even, as 2 trytes encode one character point.
• The message field is always 2187 trytes long.

Therefore, when you try to decode the message by transactions, one tryte is missing for the second transaction: it is the last tryte of the first transaction, which was ignored when you decoded the message of the first transaction.

A solution to this problem is not to decode the message per transactions, rather per bundles.

There is a utility method for this in PyOTA, check out Bundle.get_messages()

Of course, then you will have some overhead, because you will need to get the bundles from the address, not the individual transactions. To do so, you could:

1. Use find_transaction_objects() to fetch transactions like you already do.
2. Go through the fetched transaction objects. If it is a tail, save the bundle hash to a list for example.
3. Once you have all the bundle hashes, use get_bundles() to fetch all bundle objects.
4. Then you can use the aforementioned get_messages() to decipher messages.

Answer 2

I am not a developer, so I can not help you with the code.

One first note: Sending names, surnames and other GDPR protected information publicly over a DLT like the Tangle (or any other blockchain) is excluding your solution from being GDPR compliant as permanodes (or the blockchain) keep this information forever.

More information about Blockchain and GDPR compliance here

• The IOTA Foundation is working on a GDPR compliant Unified Identity, you can read more about it here

• BiiLabs worked on a DID that is GDPR compliant and you can find information and code here

For the kind of use cases where you send data over the Tangle, it is suggested to use Masked Authenticated Messaging (MAM). It is a data communication protocol that allows you to publish encrypted data streams, called channels, in transactions on the Tangle.

• Official documentation about MAM https://docs.iota.org/docs/client-libraries/0.1/mam/introduction/overview

Regarding the python code:

• The python examples are available here https://docs.iota.org/docs/client-libraries/0.1/getting-started/python-quickstart

• Here is the python specific workshop with examples https://github.com/iota-community/python-iota-workshop