Robert P. Madison was a young and eager man who returned from the Second World War in 1946 looking forward to a new beginning. Passionate about architecture since childhood, Madison knocked on the door of the Western Reserve University's School of Architecture that July. He was promptly denied admission because of his skin color. The school had never admitted a black student and declared that it was not about to start. Madison was not going to take rejection lying down. He was a United States soldier and more importantly he was a person. The color of his skin should not be a factor. Madison returned to Western Reserve the next day wearing his Army uniform, decorated with his Purple Heart, Combat Infantryman Badge and five combat ribbons. Not only did he serve valiantly but he was also wounded while fighting in Italy. Madison told the dean that "my blood has been spilled on the soil of Italy to make this country free." Upon seeing and hearing of Madison's service record, the dean agreed to let the young veteran into the school on the condition that he passed preliminary tests. Madison did so and was accepted into the program, where he exceeded all expectations.
Following architecture school, graduate study at Harvard, and a brief stint teaching at Howard University, in 1954, Madison returned to Cleveland to open his own office to practice architecture and engineering. It was the first firm started by an African American in the state of Ohio. In 1960, Madison and his family moved from the overcrowded Glenville neighborhood to Cleveland Heights, where he experienced more discrimination. However, this time it was due to where he chose to live, not where he wanted to go to school. Madison moved his family to a home he designed and built at 2339 North Park Boulevard in Cleveland Heights. They were not even able to purchase the property because they were black; instead, a Jewish person purchased it in their name, which was a common practice during that time. At that time there were barely any other blacks living in the suburb. However, Madison needed to move his family out of Cleveland to find a better education system for his daughter. In Glenville, students were being put into relay classes, meaning that they went to school for half a day instead of a full day due to high enrollments. Madison and his wife were both educated and they were not going to allow their daughter to get only half an education.
The Madisons encountered many acts of racial violence directed at their home during their first year of living in Cleveland Heights. Even prior to moving, Madison received a phone call from a man telling him that he did not want him to move to the neighborhood. Madison ignored the advice from the man. His neighbors even attempted to buy him out of the house. During their first year living on North Park Boulevard, rocks were thrown at their house and there was a woman who refused to walk on the sidewalk in front of their house. The Madison family dealt with those threats and continued to live on that street for forty years. Through all of their struggles, Madison remained a successful architect and principal at his firm, Madison, Madison, and Madison (later Robert P. Madison International), which had a hand in many prestigious projects including U.S. embassies, Cleveland Browns Stadium, Horseshoe Casino, the Louis Stokes Wing at Cleveland Public Library, and the Rock and Roll Hall of Fame, among many others.