In the past, the armed gangs, known locally as “bandits”, have been involved in cattle rustling and kidnappings, but a number of experts have recently warned that they could be forging ties with hardline groups in the region.
One militia member said the toll could be “higher than 23”, as a search for missing soldiers was under way.
Also on Saturday in the same area, five children were killed and six others injured when a bomb accidentally exploded, a spokesman for Katsina State police said.
It was not clear whether the explosive had been left by the bandits, the statement said.
Katsina State, where President Muhammadu Buhari originates from and where the vast majority of the population lives in extreme poverty, has become increasingly volatile in recent years.
The Nigerian army regularly raids the forests where the armed groups hide, but the number of soldiers is insufficient and villagers organise themselves into civilian militia.
In May, the International Crisis Group, an NGO, warned that the armed gangs could be developing links with groups such as Islamic State in West Africa Province (ISWAP).
The “bandits” have killed about 8,000 people since 2011 and forced more than 200,000 to flee their homes, according to an estimate by Brussels-based Crisis Group.
Al Jazeera English