Fireflies HD

The following videos show the movement of 881 iPhones in Europe in April 2011.

Since we couldn’t decide on a color scheme, we published all three:


High Resolution Screenshot


High Resolution Screenshot


High Resolution Screenshot

So, what do you think?

Posted in Result, Visualization | 47 Comments

Last database dump

Based on 1100 iPhone log donations we generated the probably last database dump.
License: Open Database License (ODbL) v1.0

Download
Torrent File
Magnet Link

Files
crowdflow.net-CellLocation-07-June-2011.gz: 31.2 MB (169 MB uncompressed)
crowdflow.net-WifiLocation-07-June-2011.gz: 492 MB (1,77 GB uncompressed)

Posted in Data, Result | 10 Comments

Deeper insights into Apple’s “GPS”

Some iPhone Apps need to know the location of your iPhone, e.g. the camera, Google maps, Facebook Places, Foursquare, etc.

“And that’s why the iPhones has GPS!”

Well, basically: Yes – but GPS has some disadvantages:

  • battery drain: GPS uses a lot of energy
  • mostly useless in towns and buildings: GPS needs to receive very weak satellite signals
  • very slow: GPS needs up to 12.5 minutes to initialize

GPS was not developed to work in a cell phone. So we need some additional technologies like assisted GPS or even better:

WPS or WiFi Positioning System

WPS was developed by “Skyhook Wireless” and uses information from surrounding wireless networks. So here’s the idea:

If you want to know your location the iPhone starts listening to the surrounding WiFi signals. Usually it receives some IDs (MAC addresses) from wireless access points nearby. In case th iPhone knows the position of these access points it can estimate its own location – in just a few seconds.

Unfortunately this system needs a map of almost all WiFi networks to work properly. That seems to be impossible but actually the iPhones are creating this map.

See also: F-Secure – Actually, iPhone Sends Your Location to Apple Twice a Day

And this is how Apples WiFi Positioning System works:

Step1: iPhones are gathering data

Every iPhone is looking for surrounding WiFi and cell networks.

(According to Apple, only if you use a location App but we could not verify this claim yet.)

The collected data is stored locally in the consolidated.db (WifiLocationHarvest) with:

  • information about the WiFi access point (MAC, channel, hidden, RSSI)
  • information about the collecting iPhone (latitude, longitude, altitude, horizontal and vertical accuracy, speed, course)
  • some additional information (timestamp, age, used App, confidence)

This is also done with data about surrounding cell networks (CellLocationHarvest and CdmaCellLocationHarvest).

We used actual data from our many donors to visualize this scanning process of an iPhone:

Here is a broader view of approximately 42 iPhones:

(Since all iPhones flush their data every 12 hours, an average dataset covers 6 hours. Since this video covers 24 hours, the shown activity corresponds to approximately 10 iPhones. Now guess, what it would look like for 10 000 iPhones!)

Step2: Apple creates a map

Millions of iPhones are collecting data about WiFi and cell networks and sending this data “back home” to Apple. Obviously Apple collects this data and tries to create a map of all known WiFi access points and cell areas.

Therefore they have to estimate the location and the size of each WiFi and cell area. If a WiFi device is found in different towns, it’s probably a mobile access point (e.g. MiFi).

We would love to visualize how Apple is try to combine the collected data to a WiFi and cell map, but unfortunately we do not have enough data about this process.

At least we could find some artifacts of Apples mapping algorithm. E.g. here in one of our earlier WiFi heatmaps of Berlin you can see a horizontal cut in the lower half of the map:

The latest version of the WiFi heatmap doesn’t show any artifacts:

But you can still see some artifacts in the latest heatmap of Germany:

Added on 05/17/2011:
But here is something interesting about the cell coverage.
On the left side you can see the position of a cell tower and its 3 cells (blue, yellow and red) with an (estimated) radius of 2km.
On the right side we added Apples estimated coverage of these three cells.

Step 3: iPhones download parts of the map

If an iPhone is in an unknown area it requests data about the surrounding WiFi and cells to improve the location accuracy. Then Apple sends the results. The query results basically look like this:


(scale unit: meters)

If an iPhone with activated location App travels through a new region, it constantly updates the local database

We’d love to continue our research.
You can support us by donating your data set.

Posted in Result, Visualization | 6 Comments

Second database dump

Based on 700 iPhone logs we generated two new database dumps.
License: Open Database License (ODbL) v1.0

Cell locations
Entries: 2.1 million
File size: 25,6 MB (167 MB uncompressed)
Download: Torrent File   Magnet Link

Wifi locations
Entries: 24 million
File size: 324 MB (1,75 GB uncompressed)
Download: Torrent File   Magnet Link

We created a chart for the number of network cells per country in the dataset to give you an overview:

Posted in Data, Result | 17 Comments

First glimpse of worldwide cell phone coverage

Thanks to 700 iPhone logs we now have data on 2.1 million mobile phone network cells.

Note 1:
The colors indicate the density of surrounding network cells. They do not show the range of the cell towers.

Note 2:
The results are preliminary since our database is based on only 700 iPhone logs … yet.
So please donate your iPhone logs too!

Posted in Result, Visualization | 11 Comments

First database dump of cell and wifi stations

250 iPhone Logs gave us locations of:

- 660.000 cell tower CellLocation.7z (8,6 MB) – rapidshare – .tsv

- 6 Mio. WiFi base stations WifiLocation.7z (89 MB) –  rapidshare – .tsv

License: OpenDatabase License (ODbL)

We need more datasets from outside the EU, please donate yours!

Posted in Data, Result, Visualization | 14 Comments

About the Java applet

Posted in Java applet | 30 Comments

distribution of wifi stations by manufacturer

Posted in Result, Visualization | 4 Comments

wifi stations in berlin

Posted in Result, Visualization | 18 Comments

First results

We used the datasets of 10 iPhones (300.000 entries) to create a few heatmaps.

Berlin,

Frankfurt,

Ruhr,

Germany

Posted in Result, Visualization | 12 Comments