In 1866, the New York and Oswego Midland Railroad was chartered under the direction of Dewitt C. Littlejohn, who envisioned a railroad serving a direct connection from the docks opposite New York City to Lake Ontario at Oswego. Construction on the line north of Middletown began in 1868 and was completed in 1873. Branches were also constructed to Ellenville, Delhi and New Berlin, New York; a branch was begun to Auburn from Norwich, but it only was constructed to Scipio Center before being sold to the Utica, Ithaca and Elmira Railroad in 1876. Access to New York City was provided by the Middletown, Unionville and Water Gap Railroad and the New Jersey Midland Railway beginning in 1872. On September 3, 1869, the NY&OM began using the Pennsylvania Railroad's station at Exchange Place in Jersey City, New Jersey, which provided its passengers with ferry access to the Cortlandt Street Ferry Depot in lower Manhattan and the Desbrosses Street Ferry. With the Panic of 1873, the company began to fold, and it severed its ties with the NJM and the MU&WG.

In 1880, O&W inherited the New York & Oswego Midland's lines. The O&W improved the main line by providing a new route to the New York City area from Middletown, New York, which extended to Cornwall on the Hudson River and then to Weehawken Terminal. This development was made possible by negotiating trackage-rights from the New York, West Shore & Buffalo Railway, later part of the New York Central system.

In 1886, O&W acquired the operations of both the Utica, Clinton & Binghamton and the Rome & Clinton railroads from the Delaware & Hudson Canal Company. By acquisition of these assets and construction of a new line to Sylvan Beach on the east shore of Lake Oneida, the O&W extended its operations into new market areas, and the Sylvan Beach Loop became a seasonally-significant corridor by providing transportation to central New York's recreational resort area.

By 1889, the O&W added two new branches, New Berlin to Edmeston, and Port Jervis to Monticello, connecting to the main line at Summitville, New York.

The most significant addition occurred in 1890, when the O&W constructed a 54-mile branch from Cadosia, New York, to Scranton, Pennsylvania, through the rich anthracite coal reserves in Pennsylvania's Lackawanna Valley. Revenues from this new Scranton Division strengthened O&W's revenues and provided the means for future improvements to the railroad. The railroad's W-in-O logo first appeared in 1892.

During the ill-fated "Morganization" of the New York, New Haven and Hartford Railroad (NH), the railroad acquired control of the O&W and installed NH president Charles Sanger Mellen as president for a year. Regulatory difficulties frustrated Mellen's plans to barter the O&W to the New York Central Railroad for concessions elsewhere.

The 1940s saw a receding of passenger service. In the early years of the 1940s, the Summitville-Kingston branch was reduced to a Sundays and holidays, summer-only service. Improved highways ended the O&W's passenger service to the resort areas of the lower Catskill Mountains (the "Borscht Belt") and lightly populated portions of Upstate New York, with the last train from Walton, New York, to Weehawken operating in the summer of 1948. The O&W's Walton-Delhi branch service, all in Delaware County, was also eliminated in this period. The last passenger train (from Roscoe, New York, just north of Livingston Manor, to Weehawken Terminal) operated on September 10, 1953.

By 1948 the operating losses had accumulated to over $38 million (over $69 million in 2024). The decrease of coal as a heating fuel for other than major power plants damaged its primary freight business, as did the end of rail transport of high-priority dairy products from Upstate New York to the Metro New York City area.

The New Haven offered to purchase the company in 1952, but later withdrew its offer, citing its own financial problems. Abandonment was loudly-protested by towns along the line, which considered unpaid back taxes as an investment in the railroad. The New York State legislature approved a $1 million aid bill, citing the O&W as essential for civil defense, but the state civil defense commission rejected it.

The bankruptcy court finally ordered complete abandonment, and the last freight train ran from Norwich to Middletown on March 29, 1957. Liquidation proceeded shortly thereafter. Three large scrap dealers bought the entire right-of-way from the bankruptcy court soon afterward, and removed nearly all of the rails and bridges in 1958 and 1959. All O&W assets were auctioned. The diesel locomotives found new owners, but most of the other, antiquated rolling stock and equipment was scrapped.

Certain sections of track serving shippers, many of which were industrial factories, were transferred by the bankruptcy court to other railroads, allowing continued rail access to the plants. The Delaware, Lackawanna and Western Railroad received track from Utica to New Hartford, New York, and track in Norwich, New York and Scranton, Pennsylvania. The New York Central took over sections of O&W New York track between Fulton and Oswego, and the tracks in Rome, Oneida, and Kingston. These transfers were approved before operations ceased at midnight on March 29, and the transfer of other sections to the Erie Railroad was approved later. A section of the track in New Hartford was still in operation in 2018 by the Northern Division of the New York, Susquehanna and Western Railway.