Is Orion-MSP Actually Robust Across Heterogeneous Tabular Distributions?

I’ve been looking into Orion-MSP, which uses multi-scale sparse attention and Perceiver-style memory to enable tabular in-context learning. It claims to generalize across diverse datasets, but I’m skeptical.

Some questions:

• Does multi-scale attention help when dataset feature spaces are mismatched?
• Is the Perceiver-memory robust to shifts in feature distribution or sparsity?
• What kind of datasets would actually benefit from this architecture?

If anyone has seen examples of tabular models holding up across wildly different dataset structures, I’d love to hear about it.

(Links can be shared in the comments.)