FoMo lecture series: 20 June 2023
The FoMo lectures is a series of seminars highlighting the work being done in Amsterdam around foundation models: large, highly re-usable machine learning models trained on great amounts of data. These include large language models like GPT and generative vision models like DALLE·2, Stable Diffusion and Midjourney. The talks are held alternately at the UvA and VU, and feature speakers from a variety of domains. They are intended for a technical audience.
Event details from: FoMo lecture series: 20 June 2023
- Date
- 20 June 2023
- Time
- 12:00 – 13:00
Location:
L3.36, Lab42, University of Amsterdam
About the lectures:
A Practical Approach to GPTx Models: Insights on Task-Specific Performance ( speaker: Selene Baez Santamaria (VU, CLTL)
Large language models have sparked extensive discussions in scientific communities and the general media over the past year. Particularly, OpenAI’s decision to make their GPTx models available to the public has enabled millions of individuals (with sufficient economic resources and internet requirements) to utilize them. Within research, their API accessibility has made it possible to effortlessly explore these model’s performance on a wide range of scientific tasks. In this talk, I will present my findings from three specific tasks: knowledge base completion, argument mining, and task-oriented dialogue incorporating subjective knowledge
Knowledge Representation: Neurosymbolic Integration of Reasoning and Learning (speaker: Francesco Manigrasso, Department of Control and Computer Engineering, Politecnico di Torino)
Neuro-symbolic integration aims to leverage the power of symbolic knowledge representation along with the learning capabilities of deep neural networks. Specifically, Logic Tensor Networks (LTNs) enable the incorporation of background knowledge through logical axioms, grounding a first-order logic language as differentiable operations between real tensors. However, only a few studies have explored the potential benefits of this approach for enhancing object detection and zero-shot learning (ZSL) classification. In this study, we focus on the subsumption of the isOfClass predicate, which plays a fundamental role in encoding most semantic image interpretation tasks. We introduce different architectures capable of merging LTNs with convolutional networks. FASTERLTN is an object detector composed of a convolutional backbone and a Logic Tensor Network, trained in an end-to-end manner. On the other hand, PROTOtypical Logic Tensor (PROTO-LTN) extends the current formulation of parametrized class prototypes in a high-dimensional embedding space. We showed how these architectures can be effectively trained in object detection and zero-shot learning scenarios respectively.The proposed formulations open up new opportunities to integrate background knowledge in the form of logical axioms to compensate for the lack of labelled examples by introducing room for improving the capabilities of the learning model with a sufficient knowledge base.