Shall finance department control procurement activity or conversely?

That would be a sort of a conflict of interest and there sould be a sort of segregation of duties to avoid any such conflict.

Usually finance can buy at very small values.

It's really funny that procurement should almost always sit in finance although it is in its nature as commercial as sales.

Finance should not control procurement - audit controls procurement! Finance is just where our beans get counted.