The Pittsburgh and Lake Erie Railroad was the creation of William McCreery, a prominent Pittsburgh businessman, merchant, and railroad builder. McCreery had suffered at the hands of Pennsylvania Railroad in a business that had a loss or failed. The Pennsylvania Railroad at the time used discriminatory rates which became a hot issue in Pittsburgh. On May 11, 1870 McCreery and ten other people filed Articles of Association with Pennsylvania Secretary of State. The stated length of the railroad was for 57 miles.

After 2 years the starting group was not very successful at raising the required funds and in 1877 many of the directors were succeeded by a new group of Pittsburgh businessmen. The new group was James I Bennett, David Hostetter, James M. Baily, Mark W. Watson, and James M. Schoonmaker, all influential.

In the spring of 1877, the first rails were laid down in Beaver Falls, which had the largest population other than Pittsburgh. The other reason for this was around February 1877 Jacob Henrici of the Harmony Society had business there. Henrici would also become a director in 1877.

On July 6, 1877, McCreery resigned and Bennett was elected to president with Jacob Henrici becoming a director. Henrici was the key due to his Harmony Society ties which was a communal religious group founded in 1805. In 1880, William Henry Vanderbilt's Lake Shore and Michigan Southern Railway bought stock to the tune of $200,000 in the P&LE. The P&LE would stay in the Vanderbilt's New York Central system until Conrail. Also in 1877, an agreement between the P&LE and the Atlantic & Great Western (Erie) and the Lake Shore and Michigan Southern Railway was reached for routing traffic at Youngstown, Ohio. The final track laying between Pittsburgh and Youngstown was on January 27, 1879.

At the opening in 1879, the P&LE was a poorly built, single track line. Fortunately for the railroad it was an immediate success and money was soon available for improvements.

In 1881, the P&LE became linked with the notorious South Pennsylvania Railroad (South Penn). This would lead to William Henry Vanderbilt to control of the P&LE as a link in the South Penn and the building of the Pittsburgh, McKeesport and Youghiogheny Railroad. The South Pennsylvania Railroad was planned to connect to the PM&Y.

Vanderbilt did this by buying Henry W. Oliver's and the Harmony Society's stock in the P&LE. Then Vanderbilt, aided by Andrew Carnegie, advanced the PM&Y all of the funds to build to Connellsville, Pennsylvania and then lease it to the P&LE for 99 years. The PM&Y in the end was the only part of the South Penn that was built, but it would be an important part of the P&LE. The PM&Y opened in 1883 and leased to the P&LE in 1884. Concurrently in 1883, to get the P&LE ready for the expected new business due to the South Penn linkage, the McKees Rocks shops were built.

The company came under more formal control in the 1887 by the New York Central Railroad. Lake Shore and Michigan Southern Railway's president, John Newell, took over as president of the P&LE in 1887. Under Newell, Reed and Colonel Schoonmaker; the P&LE would become the "Little Giant". From 1887 to 1927, the P&LE would become a heavy duty railroad, with double track all the way from Pittsburgh to Youngstown. The P&LE operated as an independent subsidiary, even after New York Central and Pennsylvania Railroad merged to form Penn Central.

The making of coke in Connellsville had been a big part of P&LE traffic, but by the early 20th century it had lessened. The development of by-products distillation processing of coke had moved to the Pittsburgh area. The P&LE then extended up the Monongahela River to Brownsville, Pennsylvania in 1901. The Pennsylvania Railroad at the same time had extended to Brownsville. Both the PRR and the P&LE had plans to extend even further up the river into West Virginia coke fields. Most likely due to the South Penn, they decided to work together by using the Monongahela Railway.

The Monongahela Railway then was extended south to Martin, Pennsylvania reaching the Kondike Coke fields. Later in 1915 it reached Fairmont, West Virginia.

When Conrail was formed, the Pittsburgh and Lake Erie Railroad again became an independent company because P&LE was owed $15.2 million by Penn Central, and operated as such until its merger into CSX Transportation (CSX).