Environmental watchdog Natural Resources Wales is failing in its duty to protect fish stocks and prevent poaching, anglers have said.
Stripping it of some of its responsibilities would improve the situation, the Welsh Salmon and Trout Angling Association (WSTAA) added.
Members are calling for a dedicated organisation for angling rights to be set up instead.
NRW said it was working closely with the angling community.
The WSTAA claimed NRW enforcers, who are meant to catch poachers, were too busy with other areas to do their job properly.
Anglers said they were especially worried because fish stocks were decreasing and needed protecting.
NRW is currently consulting on whether all anglers should be forced to release every salmon they catch from next year in a bid to stop a serious reduction of stocks in Welsh rivers.
But some feel the policy would unfairly punish genuine licence holders and still leave poachers free to do as they wished.
Rhys Llewelyn, president of the WSTAA and a former senior adviser for NRW, told BBC Radio Cymru’s Manylu programme: “Other countries such as Australia, New Zealand, Canada and America have created bodies that focus on looking after anglers’ rights and interests – and that’s their only focus.
“We have to ask the question ‘would we be better off with a body that’s responsible only for fisheries in Wales?’ And I believe that at the moment the answer is yes.”
He claimed that if he and fellow anglers reported illegal activity, “nobody comes to follow it up”.
Emyr Lewis, from Llanbrynmair, Powys, a former water bailiff, added: “People are blatantly showing what they’ve caught illegally on Facebook and they get away with it because there aren’t enough men on the rivers to catch them.”
NRW said its fisheries grant had been cut by 20% in real terms since 2013 – and income from rod licences had also decreased.
A spokesman said: “NRW staff do a great job day in day out in challenging circumstances and their passion and commitment to the Welsh environment, including fisheries, remains undiminished.
“We are working in extremely challenging financial circumstances and as a result have had to change the way we work.
“This includes taking a targeted intelligence-led approach, working closely with the angling community and our partner organisations to ensure fish stocks are protected and the impacts of illegal activity do not threaten the wellbeing of our stocks.”
Manylu is on BBC Radio Cymru at 12:30 BST on Thursday, 19 October – or listen again on the BBC iPlayer