Perhaps the main attraction to Penang is the old historic district of Georgetown. Here you'll find examples of colonial era straits architecture. Due to the many ethnic groups which settled here from the very beginning, there are examples of not only Georgian civic buildings, but also Chinese mansions, Indian temples and Achehnese mosques.
If you're into this kind of stuff, you can easily spend a couple of days looking around all that old Georgetown has to offer. At the very least, its worth an afternoon.
The place to start is the original fort constructed by Captain Francis Light soon after landing on Penang in 1786. Fort Cornwallis isn't much to look at. Only the outer walls, studded with a few cannons, remain. Inside, you can still see the magazine, a small church and some barracks. Most of the interior is taken up with a modern amphitheater used for performances on special occassions.
Across the street on the eastern side of the fort in the local Malaysian tourism office, where you can pick up brochures for walking tours of the Penang Heritage Trail and collect other information on the latest things to see and do. Next to the tourism office are a string of ticket offices where you can book an onward ferry to Langkawi, Medan and other destinations.
In the middle of the intersection near the tourism office is the Victoria Memorial Clock Tower, an ornate 60-foot high tower built to commemorate Queen Victoria's diamond jubilee.
To the west of the fort is a large open field called the padang. These flat playing fields are a common feature of British colonial cities. On the other side of the padang from the fort are the City Hall and Town Hall buildings. The Town Hall was under renovation in mid-2003.
Further west from Fort Cornwallis are the court buildings and the state museum. Most of the other notable houses and places of worship lie along the streets to the south of the 'government' district along the north shore.
Among the other 'not to be missed' places in old Georgetown are the Khoo and Cheah clan houses, the indigo blue mansion of Cheong Fatt Tze and the Goddess of Mercy Temple. Another interesting, although slightly offbeat, sight is the cemetary where Captain Francis Light, among other luminaries, is buried.
There are many other significant smaller houses and sights around the old city, such as the Hainan Temple. They are generally well signed, so that you can easily just wander around soaking up the atmosphere of this old colonial town.