Vanishing boosted weights: a consistent algorithm to learn interpretable rules

Learning compact but highly accurate models that help in human decision-making is challenging. Most such scoring systems were constructed by human experts using some heuristics. In this contribution, we propose a principled method with theoretical guarantees to learn interpretable simple rules. We introduce Vanishing Boosted Weights (VBW) approach which is a corrective fine-tuning procedure on decision stumps. It is a simple method which surprisingly was never investigated. We propose its extension, Corrective Federated Averaging VBW, that is practical in a federated learning scenario. We illustrate by our numerical experiments both on simulated and real data that the novel approaches are competitive compared to the state-of-the-art methods, and outperform them.