What you describe are RoI (Return-On-Investment) calculations. In business, for every investment you have to be sure that the value is greater than the costs, either if it is a new homepage, a new production machine or a fancy ML optimization.
So I would not say that this aspect is ML specific, even though it is very important.
The individual feature significance tests do not have any parameter, they just generate the p-values.
The only parameter that one can tune is the overall percentage of irrelevant extracted features. That is the expected FDR of the Benjamini yakutieli procedure.
So I would not say that this aspect is ML specific, even though it is very important.