The bodies of a four-year old girl, an infant boy, a woman and a man have been already been recovered. The Greek Coast Guard had earlier rescued six people.
The first two people rescued had told authorities they had been in an inflatable dinghy with another nine people when the vessel capsized as it headed to the island from the nearby Turkish coast.
Vessels from the Greek coast guard and the European border agency Frontex are involved in the ongoing search and rescue operation.
Lesvos has been the main entry point for migrants and refugees heading to Europe through Turkey and the Mediterranean Sea. After a deal was reached on March numbers have dropped from 1.740 arrivals per day on the Aegean Islands to 47 arrivals per day.
In 2015, some 3.600 people have drowned attempting the dangerous crossing by boat to Europe.
Meanwhile, 57.079 identified refugees and migrants were on the Greek territory on Wednesday while 44 new arrivals were reported in the last 24 hours.
According to the Refugee Crisis Management Coordination Body's figures, 23.250 of the refugees are in northern Greece, 10.234 are hosted in the region of Attica, 8.475 on the Greek islands, 1.713 are hosted in different areas in central Greece and 251 in southern Greece. 6.539 refugees and migrants are hosted in several facilities rented by the UNHCR, and 1.900 persons are living outside organized facilities.