Publications

2020

1. IEEE Access

   SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations

   Sebastian Mežnar, Nada Lavrač, and Blaž Škrlj

   IEEE Access, 2020

   Abs Bib HTML Code

   Learning from complex real-life networks is a lively research area, with recent advances in learning information-rich, low-dimensional network node representations. However, state-of-the-art methods are not necessarily interpretable and are therefore not fully applicable to sensitive settings in biomedical or user profiling tasks, where explicit bias detection is highly relevant. The proposed SNoRe (Symbolic Node Representations) algorithm is capable of learning symbolic, human-understandable representations of individual network nodes, based on the similarity of neighborhood hashes which serve as features. SNoRe’s interpretable features are suitable for direct explanation of individual predictions, which we demonstrate by coupling it with the widely used instance explanation tool SHAP to obtain nomograms representing the relevance of individual features for a given classification. To our knowledge, this is one of the first such attempts in a structural node embedding setting. In the experimental evaluation on eleven real-life datasets, SNoRe proved to be competitive to strong baselines, such as variational graph autoencoders, node2vec and LINE. The vectorized implementation of SNoRe scales to large networks, making it suitable for contemporary network learning and analysis tasks.

   @article{meznar2020snore,
     author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž},
     journal = {IEEE Access},
     title = {SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations},
     year = {2020},
     volume = {8},
     number = {},
     pages = {212568-212588},
     doi = {10.1109/ACCESS.2020.3039541},
   }

2. NeurIPS 2020

   Predicting Generalization in Deep Learning via Metric Learning – PGDL Shared task

   Sebastian Mežnar, and Blaž Škrlj

   2020

   Abs arXiv Bib

   The competition "Predicting Generalization in Deep Learning (PGDL)" aims to provide a platform for rigorous study of generalization of deep learning models and offer insight into the progress of understanding and explaining these models. This report presents the solution that was submitted by the user \emphsmeznar which achieved the eight place in the competition. In the proposed approach, we create simple metrics and find their best combination with automatic testing on the provided dataset, exploring how combinations of various properties of the input neural network architectures can be used for the prediction of their generalization.

   @misc{meznar2020PGDL,
     author = {Mežnar, Sebastian and Škrlj, Blaž},
     title = {Predicting Generalization in Deep Learning via Metric Learning -- PGDL Shared task},
     howpublished = {NeurIPS 2020 competition},
     year = {2020}
   }

2021

1. CN 2020

   Prediction of the Effects of Epidemic Spreading with Graph Neural Networks

   Sebastian Mežnar, Nada Lavrač, and Blaž Škrlj

   In Complex Networks & Their Applications IX, 2021

   Abs Bib HTML PDF Code

   Understanding how information propagates in real-life complex networks yields a better understanding of dynamical processes such as misinformation or epidemic spreading. With the recent resurgence of graph neural networks as a powerful predictive methodology, many network properties can be studied in terms of their predictability and as such offer a novel view on the studied process, with the direct application of fast predictions that are complementary to resource-intensive simulations. We investigated whether graph neural networks can be used to predict the effect of an epidemic, should it start from a given individual (patient zero). We reformulate this problem as node regression and demonstrate the high utility of network-based machine learning for a better understanding of the spreading effects. By being able to predict the effect of a given individual being the patient zero, the proposed approach offers potentially orders of magnitude faster risk assessment and potentially aids the adopted epidemic spreading analysis techniques.

   @inproceedings{meznar2020spreading,
     author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž},
     editor = {Benito, Rosa M. and Cherifi, Chantal and Cherifi, Hocine and Moro, Esteban and Rocha, Luis Mateus and Sales-Pardo, Marta},
     title = {Prediction of the Effects of Epidemic Spreading with Graph Neural Networks},
     booktitle = {Complex Networks {\&} Their Applications IX},
     year = {2021},
     publisher = {Springer International Publishing},
     address = {Cham},
     pages = {420--431},
     isbn = {978-3-030-65347-7},
   }

2. Complex Systems

   Transfer Learning for Node Regression Applied to Spreading Prediction

   Sebastian Mežnar, Nada Lavrač, and Blaž Škrlj

   Complex Systems, 2021

   Abs arXiv Bib HTML Code

   Understanding how information propagates in real-life complex networks yields a better understanding of dynamic processes such as misinformation or epidemic spreading. The recently introduced branch of machine learning methods for learning node representations offers many novel applications, one of them being the task of spreading prediction addressed in this paper. We explore the utility of the state-of-the-art node representation learners when used to assess the effects of spreading from a given node, estimated via extensive simulations. Further, as many real-life networks are topologically similar, we systematically investigate whether the learned models generalize to previously unseen networks, showing that in some cases very good model transfer can be obtained. This paper is one of the first to explore transferability of the learned representations for the task of node regression; we show there exist pairs of networks with similar structure between which the trained models can be transferred (zero-shot) and demonstrate their competitive performance. To our knowledge, this is one of the first attempts to evaluate the utility of zero-shot transfer for the task of node regression.

   @article{meznar2021TransferGraphRegression,
     author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž},
     title = {Transfer Learning for Node Regression Applied to Spreading Prediction},
     journal = {Complex Systems},
     year = {2021},
     volume = {30},
     number = {4},
     pages = {457 - 481},
     doi = {10.25088/ComplexSystems.30.4.457},
   }

2022

1. MAKE

   Ontology Completion with Graph-Based Machine Learning: A Comprehensive Evaluation

   Sebastian Mežnar, Matej Bevec, Nada Lavrač, and Blaž Škrlj

   Machine Learning and Knowledge Extraction, 2022

   Abs arXiv Bib HTML Code

   Increasing quantities of semantic resources offer a wealth of human knowledge, but their growth also increases the probability of wrong knowledge base entries. The development of approaches that identify potentially spurious parts of a given knowledge base is therefore highly relevant. We propose an approach for ontology completion that transforms an ontology into a graph and recommends missing edges using structure-only link analysis methods. By systematically evaluating thirteen methods (some for knowledge graphs) on eight different semantic resources, including Gene Ontology, Food Ontology, Marine Ontology, and similar ontologies, we demonstrate that a structure-only link analysis can offer a scalable and computationally efficient ontology completion approach for a subset of analyzed data sets. To the best of our knowledge, this is currently the most extensive systematic study of the applicability of different types of link analysis methods across semantic resources from different domains. It demonstrates that by considering symbolic node embeddings, explanations of the predictions (links) can be obtained, making this branch of methods potentially more valuable than black-box methods.

   @article{meznar2022OntologyCompletion,
     author = {Mežnar, Sebastian and Bevec, Matej and Lavrač, Nada and Škrlj, Blaž},
     title = {Ontology Completion with Graph-Based Machine Learning: A Comprehensive Evaluation},
     journal = {Machine Learning and Knowledge Extraction},
     volume = {4},
     year = {2022},
     number = {4},
     pages = {1107--1123},
     issn = {2504-4990},
     doi = {10.3390/make4040056}
   }

2023

1. MLJ

   Efficient generator of mathematical expressions for symbolic regression

   Sebastian Mežnar, Sašo Džeroski, and Ljupčo Todorovski

   Machine Learning, Sep 2023

   Abs Bib Code Poster

   We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.

   @article{Mežnar2023HVAE,
     author = {Me{\v{z}}nar, Sebastian and D{\v{z}}eroski, Sa{\v{s}}o and Todorovski, Ljup{\v{c}}o},
     title = {Efficient generator of mathematical expressions for symbolic regression},
     journal = {Machine Learning},
     year = {2023},
     month = sep,
     day = {06},
     issn = {1573-0565},
     doi = {10.1007/s10994-023-06400-2},
     url = {https://doi.org/10.1007/s10994-023-06400-2},
   }