The heterogeneity in recently published knowledge graph embedding models' implementations, training, and evaluation has made fair and thorough comparisons difficult. To assess the reproducibility of previously published results, we re-implemented and evaluated 21 models in the PyKEEN software package. In this paper, we outline which results could be reproduced with their reported hyper-parameters, which could only be reproduced with alternate hyper-parameters, and which could not be reproduced at all, as well as provide insight as to why this might be the case.We then performed a large-scale benchmarking on four datasets with several thousands of experiments and 24,804 GPU hours of computation time. We present insights gained as to best practices, best configurations for each model, and where improvements could be made over previously published best configurations. Our results highlight that the combination of model architecture, training approach, loss function, and the explicit modeling of inverse relations is crucial for a model's performance and is not only determined by its architecture. We provide evidence that several architectures can obtain results competitive to the state of the art when configured carefully. We have made all code, experimental configurations, results, and analyses available at https://github. com/pykeen/pykeen and https://github.com/pykeen/benchmarking.

The radiation belts of the Earth, filled with energetic electrons, comprise complex and dynamic systems that pose a significant threat to satellite operation. While various models of electron flux both for low and relativistic energies have been developed, the behavior of medium energy (120-600 keV) electrons, especially in the MEO region, remains poorly quantified. At these energies, electrons are driven by both convective and diffusive transport, and their prediction usually requires sophisticated 4D modeling codes. In this paper, we present an alternative approach using the Light Gradient Boosting (LightGBM) machine learning algorithm. The Medium Energy electRon fLux In Earth's outer radiatioN belt (MERLIN) model takes as input the satellite position, a combination of geomagnetic indices and solar wind parameters including the time history of velocity, and does not use persistence. MERLIN is trained on >15 years of the GPS electron flux data and tested on more than 1.5 years of measurements. Tenfold cross validation yields that the model predicts the MEO radiation environment well, both in terms of dynamics and amplitudes o f flux. Evaluation on the test set shows high correlation between the predicted and observed electron flux (0.8) and low values of absolute error. The MERLIN model can have wide space weather applications, providing information for the scientific community in the form of radiation belts reconstructions, as well as industry for satellite mission design, nowcast of the MEO environment, and surface charging analysis. Plain Language Summary The radiation belts of the Earth, which are the zones of charged energetic particles trapped by the geomagnetic field, comprise complex and dynamic systems posing a significant threat to a variety of commercial and military satellites. While the inner belt is relatively stable, the outer belt is highly variable and depends substantially on solar activity; therefore, accurate and improved models of electron flux in the outer radiation belt are essential to understand the underlying physical processes. Although many models have been developed for the geostationary orbit and relativistic energies, prediction of electron flux in the 120-600 keV energy range still remains challenging. We present a data-driven model of the medium energies (120-600 keV) differentialelectron flux in the outer radiation belt based on machine learning. We use 17 years of electron observations by Global Positioning System (GPS) satellites. We set up a 3D model for flux prediction in terms of L-values, MLT, and magnetic latitude. The model gives reliable predictions of the radiation environment in the outer radiation belt and has wide space weather applications.

No abstract

No abstract

One of the major and unfortunately unforeseen sources of background for the current generation of X-ray telescopes are few tens to hundreds of keV (soft) protons concentrated by the mirrors. One such telescope is the European Space Agency’s (ESA) X-ray Multi-Mirror Mission (XMM-Newton). Its observing time lost due to background contamination is about 40%. This loss of observing time affects all the major broad science goals of this observatory, ranging from cosmology to astrophysics of neutron stars and black holes. The soft-proton background could dramatically impact future large X-ray missions such as the ESA planned Athena mission (http://www.the-athena-x-ray-observatory.eu/). Physical processes that trigger this background are still poorly understood. We use a machine learning (ML) approach to delineate related important parameters and to develop a model to predict the background contamination using 12 yr of XMM-Newton observations. As predictors we use the location of the satellite and solar and geomagnetic activity parameters. We revealed that the contamination is most strongly related to the distance in the southern direction, Z (XMM-Newton observations were in the southern hemisphere), the solar wind radial velocity, and the location on the magnetospheric magnetic field lines. We derived simple empirical models for the first two individual predictors and an ML model that utilizes an ensemble of the predictors (Extra-Trees Regressor) and gives better performance. Based on our analysis, future missions should minimize observations during times associated with high solar wind speed and avoid closed magnetic field lines, especially at the dusk flank region in the southern hemisphere.

In this work, we take a closer look at the evaluation of two families of methods for enriching information from knowledge graphs: Link Prediction and Entity Alignment. In the current experimental setting, multiple different scores are employed to assess different aspects of model performance. We analyze the informative value of these evaluation measures and identify several shortcomings. In particular, we demonstrate that all existing scores can hardly be used to compare results across different datasets. Moreover, this problem may also arise when comparing different train/test splits for the same dataset. We show that this leads to various problems in the interpretation of results, which may support misleading conclusions. Therefore, we propose a different evaluation and demonstrate empirically how this helps for fair, comparable and interpretable assessment of model performance.

In this work, we focus on the problem of entity alignment in Knowledge Graphs (KG) and we report on our experiences when applying a Graph Convolutional Network (GCN) based model for this task. Variants of GCN are used in multiple state-of-the-art approaches and therefore it is important to understand the specifics and limitations of GCN-based models. Despite serious efforts, we were not able to fully reproduce the results from the original paper and after a thorough audit of the code provided by authors, we concluded, that their implementation is different from the architecture described in the paper. In addition, several tricks are required to make the model work and some of them are not very intuitive. We provide an extensive ablation study to quantify the effects these tricks and changes of architecture have on final performance. Furthermore, we examine current evaluation approaches and systematize available benchmark datasets. We believe that people interested in KG matching might profit from our work, as well as novices entering the field. 4

The spatial distribution of energetic protons contributes to the understanding of magnetospheric dynamics. Based upon 17 yr of the Cluster/RAPID observations, we have derived machine-learning-based models to predict the proton intensities at energies from 28 to 962 keV in the 3D terrestrial magnetosphere at radial distances between 6 and 22 RE. We used the satellite location and indices for solar, solar wind, and geomagnetic activity as predictors. The results demonstrate that the neural network (multi-layer perceptron regressor) outperforms baseline models based on the k-nearest neighbors and historical binning on average by ∼80% and ∼33%, respectively. The average correlation between the observed and predicted data is about 56%, which is reasonable in light of the complex dynamics of fast-moving energetic protons in the magnetosphere. In addition to a quantitative analysis of the prediction results, we also investigate parameter importance in our model. The most decisive parameters for predicting proton intensities are related to the location—Z geocentric solar ecliptic direction—and the radial distance. Among the activity indices, the solar wind dynamic pressure is the most important. The results have a direct practical application, for instance, for assessing the contamination particle background in the X-ray telescopes for X-ray astronomy orbiting above the radiation belts. To foster reproducible research and to enable the community to build upon our work we publish our complete code, the data, and the weights of trained models. Further description can be found in the GitHub project at https://github.com/Tanveer81/deep_horizon.