Development Update 1

By Storj

Tue, Oct 28, 2014

The following are progress updates from our devs on Storj development:

- Will
The core software for farming continues to move forward. We now have a working prototype node and farmer which can communicate with each other and prove existence of a randomly generated test file on a regular interval.

The core components of the system include the downstream-node which dishes out contracts to farmers who request them, issues new challenges when old challenges are answered and expire, and verifies challenge responses. Downstream-farmer is currently a command line interface application which connects with a specified node, requests a contract, and then begins to answer challenges.

The underlying algorithms for file storage proof are based on well-established academic literature in the area of proof of data retrievability. There are two available algorithms, one, which we call the Merkle scheme, which is more appropriate for very large files or files with shorter lifetimes, and another, the Swizzle scheme, that is appropriate for smaller files or files with a longer lifetime.

Merkle is based on creation of a Merkle Tree of file proofs. Before being stored on the network, the node prepares each file. For each file, many secret hashes are calculated, which are simply HMACs (Hash-based Message Authentication Codes). The basis of each of these secret hashes is a stream of pseudo-random numbers which can be reproduced by the node with one secret. The secret hashes are unknown to the farmer. The node then forms a Merkle Tree of these hashes, and stores for itself the Merkle Root of the file, and the secret which generated all the secret hashes. Then the node sends the merkle tree (minus the secret hashes that formed it) and the file to the server to be stored. Any time the node wants to verify the existence of the file, it simply sends the farmer one of the pseudo-random numbers, and then the farmer generates the HMAC from the file. It then sends back the whole branch of the Merkle tree and the HMAC. The node can easily verify the tree is correct, and in so doing, verifies that the farmer is storing the file. The disadvantage of this method is that the number of challenges are limited, since they must all be prepared in advance. Furthermore, for small files that will be around for a long time, the farmer must store a large amount of information (the entire Merkle tree), which gets larger when more challenges are desired. However, for large files, this algorithm may be appropriate since the amount of data that needs to be stored is relatively small.

Swizzle is based on some work by Shacham and Waters in the area of remote data possession proofs. The concept uses the properties of large integer mathematics to prove that a remote file exists. A pseudo-random function is chosen which has a secret key that only the node knows. The file is split into many small chunks, and then chunk_ tags _are generated by adding a multiple of each chunk to successive random numbers from the pseudo-random function. The chunks and tags are stored at the farmer. When the node wants to verify the existence of the file, it sends another key which is used to form a second stream of pseudo-random numbers. The farmer sends back a response which the node can verify using the secret key and the second stream of numbers. The advantage of this is that the responses are small, and also the number of challenges is not restricted. The disadvantage is that for large files, more space must be taken since the file tags must also be stored. At present, the tags are approximately 10% of the file size, but this can be reduced at the cost of added communication complexity. For more details please see the paper this work is based on, Compact Proofs of Retrievability. We are using the privately verifiable scheme.

- Josh
Instead of a separate Uptick service, as described in the last update, we decided to integrate the initial uptime features into the DriveShare software directly (specifically, the Downstream libraries that Will is focusing on – see above). I put together a public dashboard/visualization that talks to the downstream public API so you can see which farmers are active on our test verification node.

- Brian
The florincoin feature is working in metachains, passing its unit test suite. I have made a handful of small changes to web-core in order to accept the new metachains. Now focusing on integration testing with the entire MetaDisk suite and florincoind.

DriveShare
Farming tests are in progress, no coins are being earned for the moment but you can see our new farmer uptime dashboard in the picture below!

image

Early Access Plans for DriveShare Beta Released
A few days ago, we released the Early Access Plans for the DriveShare Beta, as announced on our forum and social media. Crowdsale participants get exclusive access to early farming software and applications.

Test Group A and B will be open only to crowdsale participants. Anyone can acquire 10,000 SJCX to participate in Test Group C.

There will be 3 Test Groups:

  • Group A Crowdsale participants holding over 10,000 SJCX - Not Rewarded (Late 2014)

  • Group B All participants in the crowdsale - Rewarded (Early 2015)

  • Group C Anyone holding 10,000 SJCX or more - Rewarded (Early 2015)

SJCX can be purchased at any time and be added to your address balance to be able to qualify for Test Group C. Also your original crowdsale address can be topped up to and over 10,000 SJCX to be eligible for Test Group A.

“Not Rewarded” means Group A will only be helping with testing and will not earn coins for sharing their storage. “Rewarded” means you will earn SJCX to share your free storage during beta testing (Groups B and C).

At the time of testing, make sure you use a SJCX or Bitcoin address where you control the private keys; we do not recommend using an address from an exchange. There may be a need to verify you actually own the address before you are allowed to be a tester. We are still finalizing the details and we will make more announcements when closer to the early release.

In the interest of clarity, if you have any questions or something isn’t clear please ask by posting on our Q&A on StorjTalk and we will keep it in consideration when we get closer to release.

More info here: http://storj.io/earlyaccess.html

New Storj.io Website Design
Our website Storj.io has been updated recently with a new design, quotes from great people and a new early access page. We will be adding more content to our website in the following weeks, be sure to check the site often.

image

**Community Hangout and IRC Chat
**We hosted a live chat with the Community on IRC and Google Hangouts on Sunday, October 19. There was an incredible number of participants and we want to thank everyone who took the time to join us. We received quite a few questions from community members in response to the Hangout. The Hangout Video is available on YouTube and the IRC chat log is available on StorjTalk.

Tipping Bot
We tested the IRC tip bot (tipsjcx) during the Community Chat and it worked fine to send a few tips. There was a hard limit for the total number of SJCX addresses that could be created at the time so we have now increased it. Work is in progress to finalize it. We have some work being done on a Reddit tip bot, too. We will keep you updated with progress on our Forum.

Storjcoin X Engaging with Exchanges
We need the community to show their support for Storjcoin X (SJCX). We want everyone to be able to trade SJCX on their favourite exchanges. We have a thread on StorjTalk with suggestions on how to ask for listing SJCX on these exchanges. Everyone is welcome to post tips about their favourite exchanges.

  • Voting has started for SJCX to be added on BTER.com, we only just started and are already at number 53.

  • Voting has also started on HitBTC, if you register for an account, your vote will count 10 times as much as without registering.

  • Voting is still ongoing on Cryptoine.com, we are presently at number 1.

SJCX is currently listed on the below exchanges:

You are invited to keep voting every day and show your support for Storjcoin X, the “Cloud Storage Coin”.

Interviews
The latest interview with Shawn Wilkinson on “Decentralize” is available on SoundCloud.

**Storj in the News
**Bitcoin News (Japan) - October 8, 2014
分散型ファイルストレージサービス「Storj」とはなにか
Bitcoin Magazine - October 14, 2014 
Storj Crowdsale Conclusion
Coins Source - October 15, 2014
7 Million Dropbox Credentials Leaked: Future of Storj & Snowden
IHaveBitcoins - October 21, 2014
Storj demonstrates how block chain tech can plug leaks in private cloud storage services
MaxKeiser.com - October 21, 2014
Storj demonstrates how block chain tech can plug leaks in private cloud storage services
Crypto Currency Rag - October 22, 2014
Counterparty Financial Tool
CryptoCoin News - October 28, 2014
DECENTRALIZED STORAGE APPLICATION STORJ GETS A NEW LOOK AND OPENS CLOSED BETA ACCESS TO YOU

More Posts

...
Building Tardigrade Connectors

Connectors bridge the chasm between the applications we use every day and the un...

...
Use Cases for the Decentralized Cloud

Have you heard the news? Tardigrade is in early access production, which means t...

...
Announcing Early Access For Tardigrade

Today our team is thrilled to announce our Tardigrade decentralized cloud storag...