Samuel Colt (1814-1862) received U.S. Patent No. 138 in 1836 for the first revolving cylinder pistol and along with other investors founded the Patent Arms Manufacturing Company of Paterson, New Jersey. Due to small sales that business closed in September of 1842. Still, Colt guns proved popular during the Mexican War and with the Texas Rangers. General Zachary Taylor, who was commanding troops in Texas in 1846, wanted 1000 Colt revolvers. Samuel Colt made an agreement with Eli Whitney, Jr., the Connecticut contractor for Army muskets, to manufacture the guns.

In 1847 Colt borrowed money from his banker cousin Elisha Colt and other Hartford businessmen to lease a factory on Pearl Street in Hartford, where he adapted the system of interchangeable parts to the mass production of guns. In 1851 Colt's exhibited guns at London's Crystal Palace Exposition and two years later opened a branch there that operated until 1857. The Colt's factory in Hartford manufactured 3,000 Dragoon pistols by the end of 1850, first at the Pearl Street location and then on Grove Lane.

In 1851 Samuel Colt bought property in Hartford's South Meadows where he built the Colt's Armory that was completed in August of 1855. The Armory was topped with an onion-shaped blue dome on which stood a rampant colt cast from bronze. Samuel Colt died unexpectedly in January of 1862. A fire in February of 1864 destroyed one half of the Armory and the office. Some suspected that Confederates started the fire. The armory had been running at full capacity to supply the Union army with guns, its total wartime production totaling 378,000 revolvers and 114,000 muskets.

Samuel Colt's widow Elizabeth ordered the Armory rebuilt exactly as it was. Construction was completed in 1867. In 1901 Mrs. Colt sold the company to Armstrong & Schirmer, a New York financial house. A holding company was formed in New York State although the Armory was still known as Colt's Patent Firearms Manufacturing Company.

Colt's produced the first machine gun, the Gatling, in 1867. Dr. R.J. Gatling invented the gun first fired at Confederate troops in 1862. Gatling later moved to Hartford after improving the design.

Colt's entered into association with John Moses and Matt Browning in 1891 to manufacture their machine gun. In 1895 John Browning test fired his first automatic pistol for Colt's. The Brownings reached a settlement with Colt in 1903 that gave the company all the benefits of the automatic pistol and automatic machine gun patents. The company also obtained rights to the English Vickers-Maxim automatic machine gun, giving it control over all machine gun production in the world. In 1911 Colt's gave Fabrique Nationale of Liege an exclusive license to sell Browning's automatic pistol in Europe, outside of England.

Soon after the United States entered World War I, John Moses Browning returned to Colt's and developed more powerful machine guns than his earlier inventions. In addition to machine guns, pistols were also in demand. The Army had adopted the Colt .45 as its standard sidearm in 1911. During the war Colt's produced 425,000 automatic pistols, 151,700 revolvers, 13,000 Maxim-Vickers machine guns and 10,000 new Brownings.

After the war Colt's diversified into manufacturing other products and established divisions for dishwashers, electrical, and plastics.

Colt's production dropped during the early years of the Depression but improved starting in 1933. However, on March 13, 1935 over a thousand workers walked off the job to begin a strike that lasted thirteen weeks. The company experienced further catastrophe when the Flood of 1936 and Hurricane of 1938 damaged the armory. Despite increasing output at the start of World War II and winning the Army-Navy "E" award for outstanding production in 1942, Colt's began to experience financial losses starting in July of 1943 due in part to its failure to adopt modern manufacturing techniques, resulting in layoffs.

Control of the company changed hands a few times and eventually became a subsidiary of Penn-Texas. That holding company collapsed in 1958 and what remained became the holding company Fairbanks Whitney. By 1960 manufacturing operations moved from Hartford to West Hartford.