If you go back to the time of J.P. Morgan, the world of high finance was completely wholesale. The prestigious investment banks on Wall Street appealed exclusively to large corporations, governments, and to extremely wealthy individuals.
In the 1920s, Wall Street was a world that was really dominated by professional speculators and stock pools. These people had a monopoly over information.
The history of Wall Street is inseparable from New York.