To calculate the average salary of a group of people anonymously:

Person #1 invents a (high enough) random number and adds his salary to it. He passes the sum to person #2. #2 adds his salary and passes the new sum to person #3. And so forth, until #1 eventually gets the last sum from the last participant.

Person #1 subtracts the initial random number and divides the result by the number of people. The result is the average salary.

Caveats: People can lie and the number of participants should be big enough. (With 2 people person #1 will just find out the other persons salary.)

From the “what you can do with your own email-server”-department: Here is a minimalist way to track followers on Twitter.

Bascially, I wondered how much it would take to get a daily email that lists the people that have subscribed to and unsubscribed from my Twitter feed.

To give you an idea how little it takes, here’s the script that can be executed via crond every day:

#!/bin/bash
DIR="<SOME_DIRECTORY>"
SUBJECT="TWITTER-DIFF"
RECIPIENT="lz"
EMAILMESSAGE="./msg.txt"
DATE=`date`
diff $DIR/followers.txt.prev $DIR/followers.txt > diff.txt
cp $DIR/followers.txt $DIR/followers.txt.prev
t followers @lukaszielinski > $DIR/followers.txt
echo -e "Twitter Followers $DATE\n\n" > $EMAILMESSAGE
echo -e "Diff:\n\n" >> $EMAILMESSAGE
cat diff.txt >> $EMAILMESSAGE
echo -e "\n\nFollowers:\n\n" >> $EMAILMESSAGE
cat $DIR/followers.txt >> $EMAILMESSAGE
mail -s "$SUBJECT" "$RECIPIENT" < $EMAILMESSAGE

Because I use sferik/t (line 9 above) to get the list of followers, this particular script requires ruby to run. A Twitter-API key is also required and can be obtained with a few clicks at twitter.com.

The nice thing about this ugly script is actually just the last line: Because “lz” is a user on the server a simple call of “mail” is all it takes to deliver the data. My mail-client delivers it to my desktop using IMAP.

The result isn’t pretty, but it works.

Of course with some work I can write all sorts of fancy analytics and information into my daily email. But I’ll leave this as an exercise for the so inclined reader. ;)

UPDATE (Dec. 12th 2013): “Emailserver Page Is Offline”

I just launched eigener-emailserver.de.

It’s a little offering I want to make for everyone that would like to have their own dedicated email server, beause Gmail, Hotmail, Yahoomail etc. have become increasingly unsafe.

The idea is that you rent a¡dedicated or virtual Ubuntu server in the cloud and give me your password and 50 bucks. I will log on and install and configure your new email-server. Afterwards you check if it works and change the password.

And then you have your own email-server with as many email accounts as you like. With your own domain-name of course.

Anyway, I am offering this now, and I wonder if anyone is interested.

More to come.

In a post called “The Slippery Slope” the author talks about shady tactics by online-businesses. It’s mostly about UX that tries to hide/obscure certain things from the users. Things like “Sign up for our newsletter”, “buy a subscription” or “enable ad-tracking”.

The examples are great and it’s worth your time to read it.

As with most scams, some consumers will say “That’s nothing new. This has always be happening. Think about Nokia-ringtones in the 90s that would come with a subscription.”

True, but this is a nice reminder. And good examples make consumers more alert the next time they sign up for something.

Btw. even Apple isn’t excluded. They hide their “Ad tracking”-option in some obscure settings-submenu using unnecessarily indirect language in the option-label. Quite shady, Apple!

Here’s another business idea.

Now we know that our governments are spying on us. They have the capability to intercept and store practically all of our internet traffic, including emails and phone calls. Perhaps it’s time to play around with other ideas concerning massive surveillance.

The business I imagine is a sort of a “camera club”. Users can buy surveillance cameras and connect them to modems. These modems are set up to transmit live video to the camera club, twenty-four hours a day. Each of them is just filming the street. Perhaps looking out of the window of the camera club member’s home.

The data is of course accumulated and analyzed. Useful information is automatically extracted.

The camera club automatically collects things like biometric data (including face recognition) or license plate numbers of cars. Everything is automatically parsed and stored in a database. The data gets packaged and sold to private businesses and government agencies for a nice profit.

The users that hook up the cameras to their modems get to access all the camera data for free in exchange. For example, each of them can download and browse any other user’s video data from, say, the last 24 hours. They also get reminders to reset their modems or check their connections in case their camera goes offline.

All this could lead to a near perfect coverage of all streets in realtime. For bootstrapping one quarter of a city, preferably some place with elevated street crime (for easier acceptance), would be sufficient.

Of course, I wouldn’t want the camera club to exist. I don’t want to live in a country where I’m filmed every moment of the day when stepping outside the building. But I won’t be holding my breath that a company like this won’t happen.

Everybody (governments and service providers) seem to be lying right now. For what it’s worth, the camera club would seem more honest.

It’s been a while since I looked at how public-key encryption actually works. I thought I need to refresh my knowledge, so here’s a quick summary of the RSA algorithm.

The Idea

The idea is that there are two keys, one public and one private. If I wanted to send you an encrypted message, I would encrypt it with your public key. But using the same public key, I could no longer decrypt it.

Only you can do it, with the private key which, of course, is private and only in your posession.

The public and the private key are just very (very very) big numbers.

It’s impossible (or rather completely, totally unlikely) to calculate the private key from the public key.

Mathematically speaking..

Encryption is, basically, just raising a number (=message) to an exponent (public) modulo some value (public).

Decryption is, basically, just raising a number (=encrypted message) to an exponent (private) modulo some value (public).

It just works with numbers, but any data is bascially a number. If you took this blog post, it would be just one veeeeery long number that, given the encoding (= meaning), represents this post.

So for encryption we have

    c = m ^ e (mod n) 

And for decryption it’s

    m = c ^ d (mod n)

where

    m = plaintext
    c = ciphertext
    e = public exponent
    n = public modulus
    d = private exponent

and the sign = in the above two equasions is actually not “equals to” but “congruent to”, since we are doing modular arithmetic.

That’s it.

The only difficulty is in calculating the keys. the random numbers p * q that make up the product n have to be very large for the cipher ot be secure. And calculating the secret exponent d is a matter of finding the multiplicative inverse for e “under” modulo(phi(n)).

That sounds weird, but it’s very similar to the reciprocal that everyone knows, namely the one, where multiplying the number and its inverse yields 1. For example 5, where the multiplicative inverse is 1/5, because 5 * 1/5 = 1. Same here, except we’re doing modular arithmetic, and we’re dealing with large numbers.

Ruby implementation

Here’s some ruby code that generates keys and uses them to encrypt and decrypt a message.

Note that this only demonstrates the principle. It is probably not entirely secure and definitely not very useful. (See discussion below)

But it runs and you can try it out. :)

class Integer
  # This method to check for primes is not 100% reliable, but almost.
  # Advantage: speed
   def prime?
     n = self.abs
     return true if n == 2
     return false if n == 1 || n & 1 == 0
     return false if n > 3 && n % 6 != 1 && n % 6 != 5

     d = n-1
     d >>= 1 while d & 1 == 0
     20.times do
       a = rand(n-2) + 1
       t = d
       y = Integer.mod_pow( a, t, n )
       while t != n-1 && y != 1 && y != n-1
         y = (y * y) % n
         t <<= 1
       end
       return false if y != n-1 && t & 1 == 0
     end
     return true
   end

   # a^b mod c , using much quicker squaring-method
   # check out https://en.wikipedia.org/wiki/Modular_exponentiation
   # for more information.
   def self.mod_pow( base, power, mod )
     res = 1
     while power > 0
       res = (res * base) % mod if power & 1 == 1
       base = base ** 2 % mod
       power >>= 1
     end
     res
   end
end

class RSA
  # E is the public-exponent, must be positive and smaller than φ(n) = φ(p * q)
  # greatest common divisor of e and φ(n) is 1 (= they are coprime).
  # usually 65537, because we use binary exponentiation and this number is prime
  # and only has two 1's in binary representation (= less cpu-time).
  E = 65537

  class << self

    # Returns the public modulus, the public exponent and the private key.
    def generate_keys( bits )
      n, e, d = 0
      p = random_prime( bits )
      q = random_prime( bits )
      n = p * q
      d = get_d( p, q, E )
      [n, E, d]
    end

    # Encrypts a message with the public modulus (of the receiver).
    # First encode string as a (large) number.
    def encrypt( m, n )
      m = s_to_n( m )
      Integer.mod_pow( m, E, n )
    end

    # Decrypts using the private exponent
    def decrypt( c, n, d )
      m = Integer.mod_pow( c, d, n )
      n_to_s( m )
    end

    private

    # Convert number to string
    def n_to_s( n )
      s = ""
      while( n > 0 )
        s = ( n & 0xFF ).chr + s
        n >>= 8
      end
      s
    end
    
    # Convert string to number
    def s_to_n( s )
      n = 0
      s.each_byte do |b| 
        n = n * 256 + b 
      end
      n
    end

    # Generate a random number and check if
    # it's prime until a prime is found.
    def random_prime( bits )
      begin
        n = random_number( bits )
        return n if n.prime?
      end while true
    end

    # Concatenate string (begins and ends with 1)
    # to get desired length and an uneven value.
    def random_number( bits )
      m = (1..bits-2).map{ rand() > 0.5 ? '1' : '0' }.join
      s = "1" + m + "1"
      s.to_i( 2 )
    end

    # Euler's totient function, φ(p,q)
    # needed so a multiplicative inverse (private key)
    # can be calculated. https://en.wikipedia.org/wiki/Euler%27s_totient_function
    def phi( a, b )
      (a - 1) * (b - 1)
    end

    # Check out https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm
    # for the maths.
    def extended_gcd( a, b )
      return [0,1] if a % b == 0
      x, y = extended_gcd( b, a % b )
      [y, x - y * (a / b)]
    end

    # Calculate the multiplicative inverse d with d * e = 1 (mod φ(p,q)),
    # using the extended euclidian algorithm.
    def get_d(p, q, e)
      t = phi( p, q )
      x, y = extended_gcd( e, t ) 
      x += t if x < 0
      x
    end
  end
end

# (n,e) = public key
# (n,d) = private key
n,e,d = RSA.generate_keys( 256 )

# Something we can't possibly say in public
m = "The Fast And The Furious is a great movie!"

puts "public exponent: %x" % e
puts "public modulus : %x" % n
puts "private exp    : %x" % d

puts ""
puts "Message        : %s" % m
puts ""

c = RSA.encrypt( m, n )

puts "Encrypted      : %x" % c
puts ""
puts "Decrypt again  : %s" % RSA.decrypt( c, n, d )

Discussion

So the above code implements RSA. However, there are some downsides to this implementation.

1. The message cannot be longer than the key.
2. The cipher can probably be broken.

Actually, both problems are addressed with the same solution. (And please, take this with a grain of salt, I am a total amateur just scratching the surface here.)

What is missing here is a formatting or padding scheme. If we want to encrypt long messages (arbitrary length) we need to be able to split them up so they are in chunks that can be encrypted.

Also, we need to harden the algorithm against attacks. One such hardening measure is adding random noise to each block, which is used as padding. The result is that the ciphertext will be completely different each time, but the receiver, since he knows how to apply (or un-apply) the padding, will be able to remove it again.

These things are complicated, but the core algorithm is easy. However, they are necesary to avoid certain weaknesses. That’s why, generally, just because a software encrypts data, it does not necessarily mean they do it right. You have to know what you do and/or use well known and well tested libraries. (OpenSSL comes to mind.)

As you can see there’s also some “trickery” involved. And I just mean that you need to use special algorithms, because you can’t just tell the computer to take a number with 500 digits and raise it to the power of another number of 500 digits. It would typically fail or be very slow. Same thing with checking if such a large number is prime. Thus the “good enough” algorithm above.

That’s it

I don’t know. I just wanted to write this down. Just so you can see that it’s not very difficult. Even if the proofs why it’s secure, why it works in the first place, may be not.

Btw. this whole code is based on this great Github repository. This guy also has implementations of other algorithms and some great explanation docs.

Just a quick thought.

I was talking about encrypted e-mail today and I wonder if it could be a selling point for web applications.

The idea:

Pick a web-service, doesn’t matter if it’s a project management-tool, an open-source code repo, a pizza delivery service or a pornographic website. The point is that you often (possibly always, given the disadvantages of openID) still sign up with an email address. You get a confirmation mail, click the link, and they send you email-notificatitons from then on. Yadayada.

What if they sent out encrypted emails?

Doesn’t matter if it’s PGP/GPG or S/MIME or something else. I think some people would appreciate it. It would also send a positive message that the company cares about these things.

And it would help spread the technology.

Just found this fantastic video explaining the Diffie-Hellman key exchange.

What a great idea to use the colors-methaphor to explain the basic concept.

A few months ago I created an Ember.js video-tutorial. Some people were happy with it, which means I helped them to learn Ember.js. And this in turn, makes me happy.

So, because I had time today, I decided to create another one of these tutorials. This time it’s about the best web-development framework in the world, Java Server Pages.

I’m kidding. Of course it’s about Ruby on Rails.

Here is the video (click on link below for HD-link):

Click here to watch on Vimeo (HiDef-Version)

The code for the demo-application is hosted at Github.

I apologzie for the poor sound quality. I think you can still understand me though, so I didn’t record everything a second time.

Also, in the beginning I say I want to finish in 30 minutes. It’s almost twice the time.. Well, I had fun.

The NSA-story is making headlines. That is a good thing. While it may not immediately change things for the better, it’s important that these issues are in the minds of the people so we can all reflect on them. Because they are important.

They probably matter more to people like programmers. (As Paul Graham pointed out somewhere: The personality traits that make a good programmer seem to be the same personality traits that make one question authority.) But ultimately, as we are heading for a highly technological future with interconnected devices becoming part of our daily routines, these issues matter to everyone. And will increasingly continue to do so.

First things first, though.

Yes, Edward Snowden should be considered a hero. He not only stood up for what he believed is right - at the cost of probably throwing away any chance of having a normal life for himself. He also raised a hand and spoke up in advocacy for something that is a very important cornerstone of a democracy. The ability for the people to check on what their government is doing and, like a referee, being able to call a foul-play.

And since democracy, at least in my view, is a very important achievement of human civilization, any energy invested in upholding it, is energy well spent. That includes what Mr. Snowden did. That includes the media reporting on it. And that includes the time spent by everyone thinking about these things. In order to answer a few questions for themselves.

Let’s ask ourselves some questions

For example:

• Should the government be able to pass laws in secret in a country that considers itself a democracy?
• Should the government be able to know as much as private companies? [2]
• What do we have to gain if secret organisations spy on citizens on their behalf?
• What do we have to lose?
• What does history tell us about government spying on the people? [3]
• Can we still trust internet service providers such as Google with our email?

And so forth.. Everyone can ask his or her own questions and try to come up with answers.

What Technoglogy can and can’t do

Since I am a programmer I can say something about the technological aspect of it. If we wanted to secure our data today, could we even do it? Well, let’s just take email as an example.

It should be known by now that sending an email is like sending a postcard. Everyone involved in the delivery of an email message (this includes a dozen computers owned by different organizations) can simply read that message.

Fortunately there’s a remedy for that: Encryption.[4]

Encrypting an email is a method to make sure that only the recipient of the message will be able to read its contents. That’s the equivalent of putting the letter in an envelope.

But the problem is: It’s almost impossible to do for the average person, because it simply is too complicated. You have to concern yourself with the basics of cryptography and install software that, at this point in time, still isn’t very intuitive to use.

The solution, of course, is to build products, let’s say an email client, that do provide encryption and are easy to use. (It’s not too difficult to imagine such a product.) We could have encrypted emails and most users wouldn’t even have to worry about it. It would just be, well, secure.

But would it?

Ultimately, the problem with security is that you have to trust someone. Simply because you cannot verify everything yourself. That is an impossible task.

And I am not even talking about understanding encryption alorithms (which, on a sidenote, aren’t that complicated). I am talking about the software and hardware your secure email is running on.

In the end, the security of encryption is based on keeping your (private) key secret. But if others have full access to your computer (= they can read files on it), they can steal your key and then your security is out the window.

How exactly would you know that the company that wrote the email software is not sharing your key clandestinely? How would you know your Operating System isn’t doing the same thing? Or how about the company that build your mobile phone? [5]

The answer is that, in practice, you really can’t. [6]

Other Forms of Protection

So far there has been “soft protection” in the sense that a deterrent for companies to spy on us has exited on the grounds that if they got caught, it would be bad for their business.

But if the state is passing sercret laws that force companies to spy on customers and then lie about it the protection is gone.

Meaning that we neither can have protection by technology, nor do we have protecton by business motives and/or protection by reputation.

So what is left?

The Only Viable Solution

In the end, the initiative must be directed at lawmakers. It must be directed at politicians. It must be ensured that these issues are at the core of the values in a democratic system.

We must have governments where each branch of it is accountable for its actions to some other entity. And where the checking-powers are, in fact, exercised. [7] Transparently.

In my view the problem is similar to the problem of corruption. It is very damaging to countries (from individual welfare to the economy as a whole), and very difficult to get rid of.

But, just like the surveillance state, corruption must be fought against. A battle at a time. Because the consequences can be truly horrible.

Call to Action

I encourage everyone to study the history of former soviet countries. Read books by George Orwell. Look at new democracies like Poland, Romania, Croatia. See what problems they are dealing with or have dealt with since the end of the iron curtain. Learn what it takes to become a democratic state in the sense of the EU. What independence means. What freedom means. Why the founders of the United States of America agreed on that constitution and not a different one.

Look at what is at stake here. Help to shape politics that make it difficult and illegal for the government to break important rules that justify and make up the basis for our civilizational achievements.

Finally

Edward Snowden sparked a very imporant debate. For that I want to thank him.

And now it’s time to look at America. I hope the Americans will kick some ass now. Because some asses truly deserve to be kicked. Right out of their chairs.

Notes:

[1] On Prism, partisanship and propaganda - The Guardian

[2] Should the government know less than Google? - The Economist

[3] The Lives of Others trailer - YouTube

[4] Encrypt Your Email With GPG

[5] He who smelt it, dealt it. Remember how the US government complained about the sale of Huawei phones in the USA? Because the Chinese could spy on Americans? How ironic, but not really suprising, that they would say that. Isn’t it. ;)

[6] That is a benefit of open source software though.

[7] A quick review of US government strucure