Publications
publications chronological order. Blue tags represent conference papers, pink represent journal papers, and gray represent currently unpublished work.
2024
- Quantifying Behavioural Distance Between Mathematical ExpressionsSebastian Mežnar, Sašo Džeroski, and Ljupčo Todorovski2024
Existing symbolic regression methods organize the space of candidate mathematical expressions primarily based on their syntactic, structural similarity. However, this approach overlooks crucial equivalences between expressions that arise from mathematical symmetries, such as commutativity, associativity, and distribution laws for arithmetic operations. Consequently, expressions with similar errors on a given data set are apart from each other in the search space. This leads to a rough error landscape in the search space that efficient local, gradient-based methods cannot explore. This paper proposes and implements a measure of a behavioral distance, BED, that clusters together expressions with similar errors. The experimental results show that the stochastic method for calculating BED achieves consistency with a modest number of sampled values for evaluating the expressions. This leads to computational efficiency comparable to the tree-based syntactic distance. Our findings also reveal that BED significantly improves the smoothness of the error landscape in the search space for symbolic regression.
@article{meznar2024BED, doi = {10.48550/arXiv.2408.11515}, author = {Mežnar, Sebastian and Džeroski, Sašo and Todorovski, Ljupčo}, title = {Quantifying Behavioural Distance Between Mathematical Expressions}, publisher = {arXiv}, year = {2024}, }
2023
- Efficient generator of mathematical expressions for symbolic regressionSebastian Mežnar, Sašo Džeroski, and Ljupčo TodorovskiMachine Learning, Nov 2023
We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.
@article{Mežnar2023HVAE, author = {Mežnar, Sebastian and Džeroski, Sašo and Todorovski, Ljupčo}, title = {Efficient generator of mathematical expressions for symbolic regression}, journal = {Machine Learning}, year = {2023}, month = nov, day = {01}, volume = {112}, number = {11}, pages = {4563-4596}, issn = {1573-0565}, doi = {10.1007/s10994-023-06400-2}, }
2022
- Ontology Completion with Graph-Based Machine Learning: A Comprehensive EvaluationSebastian Mežnar, Matej Bevec, Nada Lavrač, and 1 more authorMachine Learning and Knowledge Extraction, Nov 2022
Increasing quantities of semantic resources offer a wealth of human knowledge, but their growth also increases the probability of wrong knowledge base entries. The development of approaches that identify potentially spurious parts of a given knowledge base is therefore highly relevant. We propose an approach for ontology completion that transforms an ontology into a graph and recommends missing edges using structure-only link analysis methods. By systematically evaluating thirteen methods (some for knowledge graphs) on eight different semantic resources, including Gene Ontology, Food Ontology, Marine Ontology, and similar ontologies, we demonstrate that a structure-only link analysis can offer a scalable and computationally efficient ontology completion approach for a subset of analyzed data sets. To the best of our knowledge, this is currently the most extensive systematic study of the applicability of different types of link analysis methods across semantic resources from different domains. It demonstrates that by considering symbolic node embeddings, explanations of the predictions (links) can be obtained, making this branch of methods potentially more valuable than black-box methods.
@article{meznar2022OntologyCompletion, author = {Mežnar, Sebastian and Bevec, Matej and Lavrač, Nada and Škrlj, Blaž}, title = {Ontology Completion with Graph-Based Machine Learning: A Comprehensive Evaluation}, journal = {Machine Learning and Knowledge Extraction}, volume = {4}, year = {2022}, number = {4}, pages = {1107--1123}, issn = {2504-4990}, doi = {10.3390/make4040056} }
2021
- Prediction of the Effects of Epidemic Spreading with Graph Neural NetworksSebastian Mežnar, Nada Lavrač, and Blaž ŠkrljIn Complex Networks & Their Applications IX, Nov 2021
Understanding how information propagates in real-life complex networks yields a better understanding of dynamical processes such as misinformation or epidemic spreading. With the recent resurgence of graph neural networks as a powerful predictive methodology, many network properties can be studied in terms of their predictability and as such offer a novel view on the studied process, with the direct application of fast predictions that are complementary to resource-intensive simulations. We investigated whether graph neural networks can be used to predict the effect of an epidemic, should it start from a given individual (patient zero). We reformulate this problem as node regression and demonstrate the high utility of network-based machine learning for a better understanding of the spreading effects. By being able to predict the effect of a given individual being the patient zero, the proposed approach offers potentially orders of magnitude faster risk assessment and potentially aids the adopted epidemic spreading analysis techniques.
@inproceedings{meznar2020spreading, author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž}, editor = {Benito, Rosa M. and Cherifi, Chantal and Cherifi, Hocine and Moro, Esteban and Rocha, Luis Mateus and Sales-Pardo, Marta}, title = {Prediction of the Effects of Epidemic Spreading with Graph Neural Networks}, booktitle = {Complex Networks {\&} Their Applications IX}, year = {2021}, publisher = {Springer International Publishing}, address = {Cham}, pages = {420--431}, isbn = {978-3-030-65347-7}, } - Transfer Learning for Node Regression Applied to Spreading PredictionSebastian Mežnar, Nada Lavrač, and Blaž ŠkrljComplex Systems, Nov 2021
Understanding how information propagates in real-life complex networks yields a better understanding of dynamic processes such as misinformation or epidemic spreading. The recently introduced branch of machine learning methods for learning node representations offers many novel applications, one of them being the task of spreading prediction addressed in this paper. We explore the utility of the state-of-the-art node representation learners when used to assess the effects of spreading from a given node, estimated via extensive simulations. Further, as many real-life networks are topologically similar, we systematically investigate whether the learned models generalize to previously unseen networks, showing that in some cases very good model transfer can be obtained. This paper is one of the first to explore transferability of the learned representations for the task of node regression; we show there exist pairs of networks with similar structure between which the trained models can be transferred (zero-shot) and demonstrate their competitive performance. To our knowledge, this is one of the first attempts to evaluate the utility of zero-shot transfer for the task of node regression.
@article{meznar2021TransferGraphRegression, author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž}, title = {Transfer Learning for Node Regression Applied to Spreading Prediction}, journal = {Complex Systems}, year = {2021}, volume = {30}, number = {4}, pages = {457 - 481}, doi = {10.25088/ComplexSystems.30.4.457}, }
2020
- SNoRe: Scalable Unsupervised Learning of Symbolic Node RepresentationsSebastian Mežnar, Nada Lavrač, and Blaž ŠkrljIEEE Access, Nov 2020
Learning from complex real-life networks is a lively research area, with recent advances in learning information-rich, low-dimensional network node representations. However, state-of-the-art methods are not necessarily interpretable and are therefore not fully applicable to sensitive settings in biomedical or user profiling tasks, where explicit bias detection is highly relevant. The proposed SNoRe (Symbolic Node Representations) algorithm is capable of learning symbolic, human-understandable representations of individual network nodes, based on the similarity of neighborhood hashes which serve as features. SNoRe’s interpretable features are suitable for direct explanation of individual predictions, which we demonstrate by coupling it with the widely used instance explanation tool SHAP to obtain nomograms representing the relevance of individual features for a given classification. To our knowledge, this is one of the first such attempts in a structural node embedding setting. In the experimental evaluation on eleven real-life datasets, SNoRe proved to be competitive to strong baselines, such as variational graph autoencoders, node2vec and LINE. The vectorized implementation of SNoRe scales to large networks, making it suitable for contemporary network learning and analysis tasks.
@article{meznar2020snore, author = {Mežnar, Sebastian and Lavrač, Nada and Škrlj, Blaž}, journal = {IEEE Access}, title = {SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations}, year = {2020}, volume = {8}, number = {}, pages = {212568-212588}, doi = {10.1109/ACCESS.2020.3039541}, } - Predicting Generalization in Deep Learning via Metric Learning – PGDL Shared taskSebastian Mežnar, and Blaž ŠkrljNov 2020
The competition "Predicting Generalization in Deep Learning (PGDL)" aims to provide a platform for rigorous study of generalization of deep learning models and offer insight into the progress of understanding and explaining these models. This report presents the solution that was submitted by the user \emphsmeznar which achieved the eight place in the competition. In the proposed approach, we create simple metrics and find their best combination with automatic testing on the provided dataset, exploring how combinations of various properties of the input neural network architectures can be used for the prediction of their generalization.
@misc{meznar2020PGDL, author = {Mežnar, Sebastian and Škrlj, Blaž}, title = {Predicting Generalization in Deep Learning via Metric Learning -- PGDL Shared task}, howpublished = {NeurIPS 2020 competition}, year = {2020} }