{"id":11084,"date":"2025-01-10T15:46:19","date_gmt":"2025-01-10T14:46:19","guid":{"rendered":"https:\/\/dbdmg.polito.it\/dbdmg_web\/?p=11084"},"modified":"2025-03-03T12:17:57","modified_gmt":"2025-03-03T11:17:57","slug":"reading-group-17-jan-2025","status":"publish","type":"post","link":"https:\/\/dbdmg.polito.it\/dbdmg_web\/2025\/reading-group-17-jan-2025\/","title":{"rendered":"Reading Group 17 Jan 2025"},"content":{"rendered":"<p class=\" eplus-wrapper eplus-styles-uid-452832\"><strong>Paper:<\/strong> Equivariant Architectures for Learning in Deep Weight Spaces<\/p>\n\n<p class=\" eplus-wrapper eplus-styles-uid-9462ad\">\ud83d\udd17<a href=\"https:\/\/proceedings.mlr.press\/v202\/navon23a\/navon23a.pdf\">proceedings.mlr.press\/v202\/navon23a\/navon23a.pdf<\/a><\/p>\n\n\n<p class=\" eplus-wrapper\"><strong>Abstract:<\/strong> Designing machine learning architectures for processing neural networks in their raw weight matrix form is a newly introduced research direction. Unfortunately, the unique symmetry structure of deep weight spaces makes this design very challenging. If successful, such architectures would be capable of performing a wide range of intriguing tasks, from adapting a pre-trained network to a new domain to editing objects represented as functions (INRs or NeRFs). As a first step towards this goal, we present here a novel network architecture for learning in deep weight spaces. It takes as input a concatenation of weights and biases of a pre-trained MLP and processes it using a composition of layers that are equivariant to the natural permutation symmetry of the MLP&#8217;s weights: Changing the order of neurons in intermediate layers of the MLP does not affect the function it represents. We provide a full characterization of all affine equivariant and invariant layers for these symmetries and show how these layers can be implemented using three basic operations: pooling, broadcasting, and fully connected layers applied to the input in an appropriate manner. We demonstrate the effectiveness of our architecture and its advantages over natural baselines in a variety of learning tasks.<\/p>\n\n\n\n<p class=\" eplus-wrapper\"><strong>Speaker:<\/strong> Simone Monaco<\/p>\n\n\n\n<p class=\" eplus-wrapper\">&#8212;<br>\ud83d\uddd3\ufe0f<strong> <\/strong>Friday, January 17, 2025, Time 17:00-18:00<br>\ud83d\udccd Meeting Room 1 &#8211; DAUIN<br>\ud83d\udcbb <a href=\"https:\/\/polito-it.zoom.us\/j\/85220942232?pwd=7MVooHhMRAnO0Cv9dj4dhDNTQ8KGwk.1&amp;from=addon\">Zoom Meeting<\/a><\/p>\n\n\n\n<p class=\" eplus-wrapper\">Join our mailing list to receive all the updates for the upcoming reading group sessions!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\ud83d\udcc3  Equivariant Architectures for Learning in Deep Weight Spaces<\/p>\n<p>\ud83d\udcc6 January 17, 2025, Time 17:00-18:00<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"editor_plus_copied_stylings":"{}","footnotes":""},"categories":[42,45],"tags":[],"class_list":["post-11084","post","type-post","status-publish","format-standard","hentry","category-events","category-reading-group"],"_links":{"self":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/11084","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/comments?post=11084"}],"version-history":[{"count":14,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/11084\/revisions"}],"predecessor-version":[{"id":11446,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/posts\/11084\/revisions\/11446"}],"wp:attachment":[{"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/media?parent=11084"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/categories?post=11084"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dbdmg.polito.it\/dbdmg_web\/wp-json\/wp\/v2\/tags?post=11084"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}