Organizers said more than a million people flocked to Kolkata from rural Bengal for the rally which was held by 23 regional groups, including the main opposition Congress party, Asia News reported.
Modi's Hindu nationalist Bharatiya Janata Party (BJP) beat Congress and regional parties in 2014 but it faces tough challenge to stay in power after electoral losses in three key states last month.
Opposition politicians say India’s growth has slowed during Modi’s term and their first priority is to defeat him. They are taking their campaign to rural areas where anger over weak farm prices is costing the BJP at the polls.
Chief minister of the state of West Bengal, Mamata Banerjee, alleged that the “anti-people” government of Modi was nearing its end as he called on regional party leaders to join a single platform to defeat him.
"The Modi government is past its expiry date," the founder of the All India Trinamool Congress told participants from a raised platform at the rally.
The rally comes as hundreds of millions of Indian voters prepare to take part in general elections scheduled for April and May.
Banerjee is among the few opposition leaders who could emerge as a prime ministerial candidate after a poll last month showed Modi’s party could fall about 25 seats short of a majority in national elections.
In December, India’s ruling party lost power in the states of Chhattisgarh, Madhya Pradesh and Rajasthan.
Regional parties dominate the eastern states of West Bengal, Odisha, and the southern states of Andhra Pradesh, Tamil Nadu and Kerala, which together send 161 members to the 543 constituencies of parliament’s lower house.
Modi’s ruling party is also struggling to change a “perception” that it is against minority Muslims and lower-caste people.
India’s 1.3 billion people are about 80 percent Hindu and 14 percent Muslim, according to the latest census data. Backward classes make up about three-quarters of the Hindu population.
Critics say the BJP’s Hindu nationalist agenda has polarized the population.
Since Modi came to power in 2014, hardline Hindu groups and cow protection vigilantes have carried out numerous attacks on Muslims accused of eating beef or killing cows.
They have also attacked abattoirs, tanneries and leather factories which are mostly run by Muslims, indirectly also hitting lower-caste Hindus who transport cattle and work in tanneries.
847/940