This blog was originally published on October 16th, 2007. It was published on TypePad and can be found here.
Here is an article I wrote last Summer regarding the Cambridge Public Internet Initiative. I am afraid since last Summer, I have lost the thread of the project. Although I have recently become much more interested in resuming my interest. I live 3 houses down from a Cambridge Public Library branch. The branch has a WGT634U running Roofnet. I am going to break out one of my WGT634us and "mesh" it with the node at the library. I plan on upgrading the radio card (its only a mini-pci slot, so it snaps right out) to a CM-9 18dBm wireless card. To put this in perspective, the stock atheros card that comes with the WGT634U is a 60mW radio card. The CM-9 is a ~220mW radio card. Gain is logarithmic, so the extra 180mW would realize a potential gain of twice the strength and energy. I plan on coupling this radio card with a surplus directional antenna left over from Watch City Wireless. Once I am able to connect, Ill multiplex it with my Comcast cable, and use it as a backup data line.
The Original Article - Circa Summer 2006
Imagine you are an MIT student, and you can connect up wirelessly to the MIT University network from anywhere in Cambridge, MA. You can connect from any home, coffee shop or even from outside the other venerable Cambridge University Harvard. This will soon be a reality. The city of Cambridge, MIT, and Harvard University have partnered to build a city wide, open source, Linux based mesh wireless network. The Cambridge Public Internet network will provide free Internet access to all Cambridge residents, and visitors. It will allow Harvard and MIT students to connect wirelessly to their academic networks, and will provide them with pervasive Internet access throughout the entire city.
The city of Cambridge and its partners are providing funding and surplus Internet bandwidth to the project. The project uses commodity Netgear WGT634U access points, an Open WRT Linux distribution, and an open source mesh routing software developed at MIT called Roofnet. This combination of unused surplus Internet bandwidth coupled with low cost hardware and free open source software makes the project incredibly inexpensive. The city itself has pledged less than $100,000.00 to make this project a reality. This is far less than the multi-million dollar wireless rollouts in other cities such as San Francisco or Philadelphia.
It is the goal of this Muni-Uni (Municipal-University) alliance to make the Internet accessible and pervasive throughout the entire city of Cambridge. The Roofnet software classifies nodes with an Internet connection as gateways. The city will provide gateways at all 12 municipal buildings and the Universities will provide gateways on various buildings at each campus. Each gateway is an access point on top of a building with a high gain omni-directional antenna. The Roofnet mesh software running on each of these gateways connects to other nodes located throughout the city and shares their Internet connection. This combination of several (20+) Internet gateways and the mesh nature of the network will provide Internet access to all residential and business districts throughout the city.
One of the main advantages of mesh routing with the Roofnet software is the lack of user level configuration. A system administrator does not have to configure the network and the routes each time there is a change. The Roofnet software and mesh routing in particular will self create and configure a network. There is no need for an administrator to plan or design the network beforehand. Each additional node will discover its neighbors and add itself to the network. There is no need to assign IP addresses, use fancy directional antennas, or require other network users to reconfigure or change their equipment for a new node to join the network. The mesh software finds the best possible route through the multi-hop network. Loss of power, radio interference, and ISP problems can all be routed around. A self-creating, self-healing mesh wireless network will be a chief benefit to the city and its partners because of its lack of administrative overhead.
It is the job of the Roofnet software to determine the best link qualities and paths through the mesh wireless network. The Roofnet software uses the SrcRR protocol to determine the best high-throughput routes through the mesh network. This is similar to routing protocols such as OSPF. The major difference is the SrcRR protocol keeps track of 802.11 wireless link states between each of the nodes, and uses this to determine the best route. SrcRR does this through a combination of link loss monitoring, source routing, and forwarded broadcast destination querying.
When searching for a gateway to the Internet all nodes within the mesh will forward a query packet, and append their own identifier to the source packet. Once the query packet has reached the gateway node, the gateway will reply to the query packet sending the query along the same route accumulated along the path to the gateway. Because of the mesh nature of the network, an originating node may receive replies from several adjacent nodes, and each reply may have traveled across a different path back to the originating node. Each received packet creates an entry in the link state database, and the Dijkstra algorithm is run against the link state database to determine which path is the best to any given destination. Through the process of participating in the mesh network, all nodes within the network will develop adequate links based upon link state.
As the link state database is populated and equalized, the SrcRR protocol continuously measures the 802.11 loss rate of each link. Both nodes on either side of a link will continually monitor the link and report statistics back to each other. Based upon these statistics the SrcRR protocol assigns a probability as to how many times a packet will have to be retransmitted. The SrcRR protocol then combines this probability metric against the link state database, and creates a second table of the best routes through the network.
This use of a probability metric is what allows the network to function as a mesh. Once the probability of packet transmission is less than another link. The SrcRR protocol switches to the new link with the better probability. An example of this would be if a high throughput 500 kbps link were to repeatedly go up and down. LetÕs say because of interference the link were to be only functional 50% of the time. Thus the probability of transmission would be 50%. The SrcRR protocol keeps track of this metric, and would determine if there were a better quality link through the network. If there were another link that was 400 kbps it would be assigned a value of .8 (80% of the 500 kbps link) but were up and functional 75% of the time, this link would have a better packet probability metric of 60%, and the SrcRR protocol would choose it above the higher throughput lower quality link.
500 kbps (100% of the highest throughput) 50% uptime = 250 kbps or 50% packet probability 400 kbps (80% of the 500 kbps link) 75% = 300 kbps or 60% packet probability
The SrcRR routing protocol is implemented using the Click modular router. The Click router is another project created out of the CSAIL research laboratory at MIT. Click is described as a modular router, because it allows various routing components to be added to it as software modules. The SrcRR protocol is implemented as a Click software module. The various link state monitoring, and general mesh routing are all done within this Click router and the SrcRR modules. On a Roofnet node Click runs as a user level program. To run the Roofnet software there is no kernel level interaction. All you have to do is compile the Click router and Roofnet modules for your platform. Running Click as a user level program, instead of a kernel level program is very useful and beneficial. It makes it very easy to install, run, and is quite portable to other systems.
The Click router essentially does three things. It determines a route to a destination. Ensures that the each link along that route is of adequate quality, and advertises a new route if the previous one should suddenly become unusable. It accomplishes this by reading and writing the lower level 802.11 MAC layer information from the wireless network card. The Roofnet project uses a modified Madwifi driver and requires an Atheros-based wireless card. An Atheros wireless card is a requirement to run a Roofnet node and network. Thanks to the code base submitted by Atheros, the Madwifi Linux kernel driver allows the SrcRR protocol to do its low level meshing magic. In addition an Atheros and Madwifi solution support many wonderful wireless features such as virtual access points, WDS mode, and WPA security. The Madwifi drivers in combination with an Atheros card make a wonderful Linux wireless solution, but because of the Atheros open code base they are a necessity for a Roofnet access point.
The Cambridge public Internet project is using the Netgear WGT634Us as the mesh nodes for the network. The access points come with Linux pre-installed. They have a 200MHz Broadcom CPU, 32 MB RAM, 8MB of flash memory, an Atheros 802.11b/g card, two Ethernet interfaces (one for the WAN connection one for the LAN) and a USB 2.0 port. They are wonderful access points, but they are no longer being manufactured by Netgear. Becaue of this, the Roofnet researchers are developing a low cost access point to be used with their software, and are in the process of manufacturing these nodes and have developed a business to support the Roofnet software.
The Roofnet software and general information on the Netgear WGT634U can be found at the Roofnet project site pdos.csail.mit.edu/roofnet/doku.php?id=wgt6.. . The Roofnet software has gone through several versions but the latest version is now roofnet-2.0.6. To upgrade to this you first have to install roofnet-1.6.1, and then use the built in upgrade tool to upgrade to the latest 2.0.6 release. This need to install roofnet-1.6.1 first, will go away in future versions of the Roofnet software for the WGT634U.
The major difference between Roofnet version 1 and Roofnet version 2 is the inclusion of OpenWRT in the Roofnet version 2 release. Open WRT is described as a Linux distribution for embedded devices. It comes with a fully readable and writable file system, and has an advanced package management system. The version of Open WRT that comes installed with Roofnet is a 2.6 kernel, has a JFFS file system which provides persistent changes, and all the Click and SrcRR software installed and ready to go. In addition the system comes preinstalled and configured with various dameons and Linux tools such as iptables, an SSH daemon, a web server, and even Perl. Thanks to the Open WRT package manager ipkg additional software is very easy to download and install. Although you are limted by the 8 MB of persistent flash memory.
Installation of the Roofnet software onto a Netgear WGT634U access point is incredibly easy. There are several steps, but it is mostly uploading the correct firmware to the Netgear access point in the correct order.
You can install Roofnet on the Netgear WGT634U from any type of computer (Linux/Mac/XP, etc.):
- Download the roofnet-1.6.1 image file pdos.lcs.mit.edu/roofnet/netgear/roofnet-1... and the roofnet-2.0.6 binary file pdos.csail.mit.edu/roofnet/netgear/roofnet-.. to your computer.
- Login to the routerÕs admin interface at http://192.168.1.1/ (default login is ÒadminÓ with the password ÒpasswordÓ)
- Select the Router Upgrade option on the left-hand panel, then upload the image file under the upgrade file section.
- Hit the upload button and watch the status bar on the screen to check on progress.
- If you hadnÕt changed the default password, it is now r00fnet(with two zeroes).
- Load the web interface at http://192.168.1.1. You should now see a ÒUpgrade to Roofnet 2.0Ó option on the top left-hand panel. If you donÕt hit reload, or right-click on the left hand panel and hit reload.
- Upload the roofnet-2.0.6 binary file under the Upgrade to Roofnet 2.0 panel. The router will reboot twice and generate its ssh keys, which takes a few minutes.
- When it is done, youÕll see a new web interface at 192.168.87.1, and the node will advertise itself using the essid ÒroofnetÓ. The login is now roofnet and the password is r00fnet with two zeros.
All configuration of the node is done through SSH. At this time there isnÕt an included administrative web interface. To SSH into the access point bring up a terminal and type ssh -l roofnet 192.168.87.1 if you are connected via an Ethernet cable or ssh -l roofnet 10.0.0.1 if you are connected via the wireless network. At the password prompt type in r00fnet with two zeros. Once you are in have a look around. Open WRT is a fully functional Linux distribution. The /bin/mount command shows what file systems are mounted at boot time.
roofnet@OpenWrt:/# mount /dev/root on / type jffs2 (rw) none on /dev type devfs (rw) none on /proc type proc (rw) none on /tmp type tmpfs (rw,nosuid,nodev) none on /dev/pts type devpts (rw) none on /sys type sysfs (rw)
Take notice of the first entry the read write mounted jffs2 / file system. This mount point is a direct link to the 8MB flash memory installed on the Netgear WGT634U circuit board. The / mount point has a normal looking Linux file system. It has an /etc directory for configuration of the system and contains the /etc/init.d startup scripts. It has a /bin and /sbin directory for user level commands. It even has a /www directory for the built-in web server.
To change the default password from r00fnet with two zeros simply run the command passwd roofnet. This command will prompt you for a new password for the roofnet user. Enter in the new password twice, and voila thanks to the readable and writable JFFS file system you have now permanently changed the /etc/passwd file and changed the default password on your access point.
After installation you can use the node as a normal access point, or you can pair it with other Roofnet nodes, and create a mesh wireless network. To use it as a normal access point, connect your Internet connection via an Ethernet patch cable to the WAN interface of the access point. You can use a wireless client such as a wireless enabled laptop or PDA to associate with the network ÒroofnetÓ and once connected surf the web, or check your e-mail. By default every roofnet node will act as an access point and a mesh node. The Roofnet software creates multiple virtual access points (VAP) using the Madwifi driver. Each virtual access point shows up in a /usr/sbin/iwconfig command and they are named ath0, ath1, and ath2. The first ath0 is the VAP running as an access point with an essid of ÒroofnetÓ. ath1 and ath2 are the interfaces used by the SrcRR protocol to communicate with other nodes within the mesh network.
ath0 IEEE 802.11b ESSID:"roofnet" Mode:Master Frequency:2.422 GHz Access Point: 00:0F:B5:3F:2E:0E Bit Rate=11 Mb/s Tx-Power:20 dBm Sensitivity=0/3 Retry:off RTS thr:off Fragment thr:off Encryption key:off Power Management:off Link Quality=58/94 Signal level=-37 dBm Noise level=-95 dBm Rx invalid nwid:31 Rx invalid crypt:0 Rx invalid frag:0 Tx excessive retries:0 Invalid misc:0 Missed beacon:0
ath1 IEEE 802.11b ESSID:"" Mode:Managed Channel:0 Access Point: Not-Associated Bit Rate:11 Mb/s Tx-Power:20 dBm Sensitivity=0/3 Retry:off RTS thr:off Fragment thr:off Encryption key:off Power Management:off Link Quality=58/94 Signal level=-37 dBm Noise level=-95 dBm Rx invalid nwid:44 Rx invalid crypt:0 Rx invalid frag:0 Tx excessive retries:0 Invalid misc:0 Missed beacon:0
ath2 IEEE 802.11b ESSID:"" Mode:Monitor Frequency:2.422 GHz Access Point: 0A:0F:B5:3F:2E:0E Bit Rate:0 kb/s Tx-Power:20 dBm Sensitivity=0/3 Retry:off RTS thr:off Fragment thr:off Encryption key:off Power Management:off Link Quality=58/94 Signal level=-37 dBm Noise level=-95 dBm Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0 Tx excessive retries:0 Invalid misc:0 Missed beacon:0
To change the essid or channel of the ath0 interface you use the command /usr/sbin/iwconfig. The format of the command is iwconfig interface [essid {NN|on|off}]. For example to change the essid from ÒroofnetÓ to Òliving_room_netÓ you would issue the command:
/usr/sbin/iwconfig ath0 essid living_room_net
Or if you need to change the channel on your node from the default channel 3 to channel 11 you can run the command:
/usr/sbin/iwconfig ath0 channel 11
To make each of these changes permanently cd /etc/init.d and open the startup script S80roofnet with vi. This is the startup script that kicks off and configures the Roofnet software. It creates and configures the ath0 interface and starts up the click router. On lines 43 and 44 of the start up script are the same iwconfig commands you ran above. Change each line to read:
/usr/sbin/iwconfig $AP_DEV channel 11 /usr/sbin/iwconfig $AP_DEV essid living_room_net
After doing so, the changes you made are now permanent and will survive between reboots.
Low cost hardware, free open source software and surplus bandwidth are all that is required to build your own city or town wide mesh wireless network. To get started on a Roofnet implementation check out pdos.csail.mit.edu/roofnet/doku.php. It is the website of the original research project, and it has information on the Roofnet software, and the Roofnet implemented hardware.
Tom is a resident of Cambridge, who is psyched that he lives next to two municipal buildings. He is also a passionate Boston wireless advocate, who looks forward to shooting a WiFi signal up the Charles River.