https://ai.googleblog.com/2021/05/do-wide-and-deep-networks-learn-same.html
Do Wide and Deep Networks Learn the Same Things?
Tuesday, May 4, 2021
Posted by Thao Nguyen, AI Resident, Google Research
“In very wide or very deep models, we find a characteristic block structure in their internal representations… Comparisons across models demonstrate that those without the block structure show significant similarity between representations in corresponding layers… These properties of the internal representations in turn translate to systematically different errors… when they are evaluated on the same test set.”