The Library was founded in February 1908 by the Ladies Literary Society and was organized as a reading room. Membership grew and on November 22, 1909 the library incorporated in the name of the Port Jefferson Free Library Association.
The Library is a Free Association Library. The term "Association Library" refers to a library established by a group of private individuals operating as an association; and the term "free" as applied to a library means one that is maintained solely as a public service for the benefit and free use of all the people of the community.
The Library, chartered to serve residents of the Port Jefferson Union Free School District #6, is a public facility that extends services and borrowing privileges to library cardholders of all Suffolk County Libraries.
In 2005 the Library building was officially placed on the National Register of Historic Places, a distinction designating it as worthy of preservation and restoration.
