Logo

Software Developers Journey Podcast

#259 Larysa Visengeriyeva from mind scripts and biases to MLOps

Transcript

⚠ The following transcript was automatically generated.
❤ Help us out, Submit a pull-request to correct potential mistakes

Larysa Visengeriyeva 0:00
And this is my advice for people who are new in ML ops. Go and find your community. And the first community which is really big community and large community is the ML ops community ml ops dot community. Join the slack, ask questions, and you will find so many the brand people there too, who will answer your your question who will who will help you in your journey. This is the way to, to navigate in in any new topic in the tech world or beyond the tech world. So find the community, like minded people, join them, and enjoy the life.

Tim Bourguignon 0:53
Hello, and welcome to developer's journey, the podcast bringing you the making of stories of successful software developers. To help you on your upcoming journey. I'm your host team building. On this episode, I received Larissa visiting Grijalva. Lisa received her doctorate in augmented Data Quality Management at the Technical University in Berlin, Germany. She is a technology consultant that you know Q, where she focuses on ml ops, that's machine learning operations. These are architecture and domain driven design. She is also the founder of the woman plus in data and AI Festival, which will happen in Berlin, right after the show airs actually. And she is generally very active in the tech communities here in drug. Teresa, welcome to the afternoon.

Larysa Visengeriyeva 1:43
Hi, Tim. And thank you very much for having me.

Tim Bourguignon 1:47
It's my pleasure. Just just to tell you, we've been laughing for 40 minutes before this, which is a recording. Usually, we go for 20 minutes most but we've been having such a great time. So I'm really looking forward to this.

Larysa Visengeriyeva 1:59
We have to record that.

Tim Bourguignon 2:03
We should have we should have it. But before we come to your story, I want to thank the terrific listeners who support the show every month, you are keeping the dev journey lights up. If you would like to join this fine crew and help me spend more time on finding phenomenal guests then editing audio tracks, please go to our website, Dev journey dot info and click on the Support me on Patreon button. Even the smallest contributions are giant steps toward a sustainable dev journey. journey. Thank you. And now back to today's guest, Larissa, the show exists to help listeners understand what yesterday looked like and imagine how to shape their own future. So as as usual on the show, let's go back to your beginnings. Where would you place the start of your journey?

Larysa Visengeriyeva 2:54
Okay, the Deaf journey goes back deeply, deeply, deeply into my childhood. Because I was at the beginning of the school, like, like, let's pull out the typical cliches I was not good at math. But then I become good at math. And, But what fascinated me most in the school when I was quite young, or like the fifth grade, so I so the books about the informatics, the computer science and what fascinated me most these books that I didn't understand anything like they were zeros and ones through the the whole book and you know all these I know them now, but Boolean tables you know, and I saw the same kinds of robots who try to explain something. So, it was fascinating because I had no idea what is this about? And then in high school when we got first hours of informatics I just I just I just got so like the computer become my friend. I could I could be creative and draw pictures just by knowing the coordinates. And and then we had some like in the school we had small task like like some cool They about some, you know, geographical stuff, just you know, write the question try to answer and so on and so on. And the first language programming language was basic. So, what I have to tell also, if you should you like to put everyone in a context, so, I grew up in a country, which doesn't exist anymore to the Soviet Union. So, and and, yes, at some point in our high school, we had the Voice of America Radio. And then in the 90s, there was a show, the show about the internet. So, imagine, so we had, we had no internet, but there was a show on the radio about the internet. And I had no idea what they are talking about, but these unknown stuff was so fascinating. So and, you know, the, like the the layer of programming, the near the limit layer of math, which I like, with the time I become really good in math, and then all this show, from Ray radio, about the internet, you know, all this, you know, shaped me. And as boring as it sounds, I had no other ways as going to the tech. So it was absolutely clear to me that I'm going to study computer science, I'm going to study informatics so whatever but but but I couldn't imagine anything else. Just just like the sort of I know it's my way from the beginning.

Tim Bourguignon 7:11
Can you make sense of that retrospectively and see what attracted you back then? Or is it still well you're attracted period

Larysa Visengeriyeva 7:24
you know, these this stuff this like mystical stuff with zeros and ones through the whole book, you know, with all that you know. Or or you know, the basic commanders were like, we have the soil the coat in the book. So, I saw the cycle so the cotton on so what is it this you know, this unknown this this unclear, but, but it was fascinating and this fascination, like a shaped me into these fields. Okay,

Tim Bourguignon 8:03
so, the past was traced, you will be going to a minute or so the math informatic and technology or stem in English, I think Oh, and that's that was the the the highway for you or did you did you double in there and try to find a different way?

Larysa Visengeriyeva 8:23
No, I had no double it was so somewhat tried. So, there is another story so I can tell this story as well. Someone tried to infer like, okay, so, you had a psychologist in the school and the psychologist in the school told me in high school, that Larissa you are leader, you are leader, you have to go you have to go and become a director or vice president of something, you know, and it was so like, straight from the school. So you know, where she is she like you have to go and become like, advocate. You have to go and become a doctor. Like, you know, all this kind of stuff, but with why so I so I have my I have my way here. So So I think I'm going to study informatics. So it's like from the beginning. And this is another another thing which fascinated me in school, which also shaped me into the computer science direction. They will see you know, the calculator the programmable calculator. So the first so when I the first time I wrote it program for solving the equations, and I installed this program on the calculator, and I saw the solution, I was thinking, wow, I don't have to think the calculator is doing for me. And this you know, this, you know this, you know, the laziness of the programmer, you know, what's clicking in my head, you know, I can I can automate stuff and I don't have to think about this. And, you know, this, this was also the experience, which I think shaped me into, into this direction. But yes, as far as I always knew, I will go into the computer science, which

Tim Bourguignon 10:49
which is awesome. Yes. How did you discover what kind of computer sciences were the right for you?

Larysa Visengeriyeva 11:48
Oh, I think you don't know. At the beginning, no one knows. Because it and computer science, software development is such a wild area, you know, so you, you actually, you can you these personalization, what I think what you're trying to ask is come so will come when you enter the job. So first with the first job with a second job. So probably, so, during, during my study at university, I knew for example, so that I won't go into robotics, or something like this. But then in during my journey during the more experienced with my PhD, so I discovered machine learning. So and that's how I come into the CIO, and the CIO, it's like, when you get more information, or you get hands on experience, so you are not reluctant into to the area. So it's a it's mostly so you Yeah, so kind of biases.

Tim Bourguignon 13:25
Did you did you your PhD at the end of your studies and the continuation? Or did you work before and then come back to doing a PhD?

Larysa Visengeriyeva 13:33
Yes. Regarding the PhD, I'm not a traditional like path going so when I so I had two degrees. So first, I have a first degree at the University in Odessa in Ukraine. So, this is master of education. Then I come to Germany. So I had I understood that I have to get a second degree because because it was a master of education in informatics. So basically, I am a teacher

Unknown Speaker 14:15
okay

Larysa Visengeriyeva 14:17
for computer science, and when they come to Germany I just realized that it's not so I want to go and I want to go and develop a software I want to go more technical so I want to and then the only way I so it's to to finish to get a degree here in Germany and I got into the excellent university in brand new book and a half. So I started there I got I got most brilliant professors there so I really happy I finished quite Soon, and immediately went to the industry and worked as a back end developer with specialization to the database, developer and.

Larysa Visengeriyeva 15:15
And having a developer This is my first journey. And after four years, after four years in the industry, I decided to go into the academia and get a PhD.

Tim Bourguignon 15:32
Why? So, obviously, without any biases in my in my, in my question, but why

Larysa Visengeriyeva 15:43
sometimes I, sometimes or during the PhD, I also asked this question. So this is the biggest question the universal What the hell you, Louisa you're doing, so why why you give up, you're safe with, you know, loving back and developer job and went into these academia. Okay. I think after years of thinking about this, I think there are social scripts or expectations from the society, which

Larysa Visengeriyeva 16:28
sometimes shape your decisions. And one of them, one of them are so aware that like, you have to go the career ladder, so and become a director, or vice president or I don't know head of something,

Larysa Visengeriyeva 16:59
otherwise you are not successful. And these kinds of scripts, which are installed into your head, they are invisible scripts. And this, but these scripts cannot be always useful for you. So, so you're influenced by such scripts? And these exactly these scripts, their career path, so as women and it, you have to get a PhD, especially in Germany, you know, to become a, you know, Vice President, what ever, you know, like, leadership position, or executive position. And, I have to admit, yes, this is a, this was the first reason why I did this move, because it was not easy. I had, I had a newborn child, and I was on maternity leave. And, you know, I was so naive to think about, yes, I can do everything. giving a birth, having a small baby, and, you know, doing a PhD at the same time, so why not? Why not? But, but when I left the corporate world, sometimes, sometimes, you know, some dots, you know, come to me so the question why I'm doing this, you know, this become more apparent. So, I, I had to, I had to decide so, because I definitely understood that career path is like this vertical career path, it's not my way, it's just not my path, someone some, all these invisible scripts. They are working, working in my head. So and I have to, like reformat my brain regarding the scripts so and destroy, as I call this destroying matrix, so or break out of matrix to see what's going on. So and then I also I had to reframe, reframe my motivation why I'm doing this. So at the same time I so at the beginning of a PhD I worked on on a natural language processing project, and so I kind I dive into these areas. And by no means it was easy journey. But what I discovered, I discovered some questions, which kind of like toggle something in me. So, and it, I realized that while working on natural language processing, some somehow the data preparation and data cleaning part is is about 80 90% of the time. And the most of the code I wrote was like the massage massage massaging the data. So, and this at this point I realized is a real question is not actually, to me, was not the natural language processing, which is also important part of the research and area. But for me, it was how can we make the data cleaning data preparation part as comfortable as possible for people who are working with data? So, and then when I asked this question, the whole motivation of doing a PhD completely, or it was like absolutely clear to me that I'm doing the PhD because I want to answer this question, I see the real problem, I saw the real problem. And I saw that we can we we still have to do a lot of in this regard, but as a contribution I saw the like, all my work or the methodologies, the methods for data preparation, and data cleaning, making as comfortable as possible. So, this was a main motivation for doing a PhD. So, like I saw, to wrap up, I started with a false motivation, I understood, I understood this, this motivation is false, then I had to decide. And luckily, at the same time, I found the real problem. And then it was not a question anymore, for me why I'm doing a PhD.

Tim Bourguignon 22:43
Thank you for explaining this, really, with the psychology aspect and then finding your your way aspect is really, really puts things into perspective. I work for a startup and we're seeing this as well in the startup world really trying with an assumption, this is going to be a problem and then you start working on it. And quite often you realize not quite and then you have to pivot you have to change and and if you do that well enough, then you will find the right problem to work on. And if you don't, you will be continuing on false assumptions and maybe go into what

Larysa Visengeriyeva 23:13
absolutely and I have for anyone who is interested in like in understanding these kinds of aspects like this is a meta level of you know, the purpose of what we're doing is the the book The Art of Doing science and engineering

Tim Bourguignon 23:42
reshard your books on yourself read now.

Larysa Visengeriyeva 23:45
Richard Hamming. So I really recommend this book and at least his YouTube video, you and your research. So it will it will be suitable also for the startup world as well.

Tim Bourguignon 24:01
What is the name of the book again?

Larysa Visengeriyeva 24:03
The Art of Doing science and engineering learning to learn

Tim Bourguignon 24:08
I the link to that so you don't have to search either the book or the other YouTube videos. Thank you both for having us. I'm glad you explained the whole process but I think from German standards, I did a full part introducing you I didn't call you Dr. Larissa and

Tim Bourguignon 24:31
the motivation who know in Germany

Larysa Visengeriyeva 24:35
I put this doctor in my in the title passport, the passport. Oh, you did? Okay. So so this it's good.

Tim Bourguignon 24:46
For some reasons the Germans are so so so hell bent on this. This is very important here in Germany. And I need to find some some statistics and how many doctors they are in Germany versus the rates for the rest of the world. I'm absolutely certain there is a There is an influence there, that this aspect on its own drives more PhDs in Germany than the rest of the world, but haven't done the work so far.

Larysa Visengeriyeva 25:08
Well, I can tell you that in especially in Germany, it's indeed makes a difference. So when I got the title, so I realized the how change the change everything, almost everything,

Tim Bourguignon 25:29
which validates at least or validates valid is not the right word. I don't I don't have a better work for now, validates this, this thinking pattern descript, you are talking about saying, Well, if you have a PhD, indeed, it will help you as a woman or not, but as in this case, even more grow into this male dominated industry, and it will help you go toward the career ladder, VP and and cetera, that you were imprinted upon? To put it in scripts way. I'm not sure if that's what you're searching for still, but at least it went into the search. And so that script was working in itself.

Larysa Visengeriyeva 26:09
Yeah, but But I, I was not doing PhD. Because of, because of this. So sure, I can, I can do. I mean, I can I can do this as well. So but but the motivation was different.

Tim Bourguignon 26:29
I understand you. Yeah. Sorry. That's why as I was just highlighting that at least, this this script was was a consequence in itself, that at least it worked. It would have been even worse, if you were not searching for this and achieving it and not having the results that you were not hoping for.

Tim Bourguignon 26:50
I think I would just add myself, I'm sure you did, but what did you do after finishing your PhD having closed this, this part of your curriculum saying well, you found an itch to scratch, you found a problem you wanted to solve? Now, you got a PhD for that, what do you do after that? How do you rebound on this?

Larysa Visengeriyeva 27:12
Also, thank you for this question. Because, because I had I found another problem.

Tim Bourguignon 27:22
Tell us about it.

Larysa Visengeriyeva 27:26
So during my PhD, you know, you so you d is when you dig so deep. In some areas, you will find, of course, you will find more problems, it's it's normal, it's, it's it's okay. And one thing I realized and this is my story about So, like a breach to the ML ops area when for my PhD I developed methods for data cleaning, and these methods were based on machine learning. So, I use machine learning to to implement some systems for data cleaning for metadata management and so on. And in the end, I looked at the code at the system as a whole and thought hmm, so, just let me say imagine you have a customer right now. And so, try to imagine shipping this mess of code, this hack to the customer, no way. So, no way then then I started to look for four methods like okay, we we have enough knowledge, enough patterns, enough best practices for software engineering, for traditional systems, but at that time, it was 2015 2016. So, at that time, I started looking for software engineering for machine learning systems, like what are the best practices for software systems, which have these machinery and components in embedded into this, because you have, you have like different lifecycle of software systems, when as soon as you have this machine learning model inside. So I found nothing or quite a few scattered stuff not not systemized not not, you know, there is was no code or term coined like ML ops and then I saw this and I said, just because software engineering, so I liked this topic, software engineer how to ship software. So how it should be done? You know? I was thinking, Yes, this is the topic I will tackle when I finished when I defended my PhD. And this is what happened. So, so this, you know, this an unanswered question back into the 2016. So, the like, so I actually, at this time, I knew already that I will work. Later, later on, I will work on this topic, and this has happened. And the, the like the Web site, which is in the meantime called Bible of ml ops, the ammo minus ops.org is is actually the result of that journey.

Tim Bourguignon 31:08
Wow, cool. I love how I'm putting people in boxes, which I always see to two different paths coming out of academia, there are people who are just so enamored by the project that they somehow will continue, and they find a way to, to bring this very niche project they've been working on for years into into the world, and you have to change the skill sets you're using, you have to think differently, you have to start teaching, start projecting this project, which is quite often very, very, very hard to understand. And, and really trying to work on that, which is very hard to to, to make this jump. And there's other paths where people just can't do that. And it stays in academia for years, and it just doesn't become reality. And I'm, I'm thrilled to see that you were able to take this subject and stay with the subject scale, to change the skill sets, you will you needed to really start bringing this into the world and make something out of it and, and be able to name a link now and say, Well, this is what it became. This is awesome. This is really cool.

Larysa Visengeriyeva 32:15
Thank you very much. So I'm you know, when I am getting quite a, you know, making the impact out also something which shapes and motivates me on my journey. And this is these, these sides, the MO opposite org is actually something what made an impact on the community. And until now, I'm getting a feedback that companies are doing onboarding of the staff with this website. And I was Wow, thank you. Really, I think you're a master I'm so glad you use it. Wow. So I love I love I love such feedback. And this feedback, you know, again, motivates me for doing more for the more into this star. So for doing for writing more about this topic, the ML ops because because ml ops by no means is, is ready. This is still developing topic. And so, yeah, so yes, these positive positive positive feedback from people from companies were motivates me so so I produce more and this so people can see and are happier, you know? So and so you see the fly flywheel effect, you know, so often motivation is the positivity of our staff.

Tim Bourguignon 34:07
So you're sticking with this topic for a while no new shiny topic that emerged in going deep and toward which you're you're you're trending.

Larysa Visengeriyeva 34:16
Okay, no, not at all. Sometimes I think. So. I had no ops. I just hated sometimes sometimes to get into guys. Okay. Sometimes become a mess because of some the mess of the tooling right now the mess of the emerging startups who are pretending to be doing tooling for ml ops and so on. But, but I think yes, but I think it's it's a topic which need to be stopped Tyst are like DevOps, like, or so my hope is that, or my, I think that in the future, the machine learning development or the machine learning models will be so prevalent in the software systems, we will have so many of them insight or systems that the ML ops in DevOps will merge and become one topic. So, but But during this journey, I, while talking to customers, I sometimes realize there are so much homework needs to be done at the companies in the industry. So, companies need to be, like, get organized with the your data. So, there's a lot of mess within companies. So, company is like organizations collecting so much data. So, at some point, so the don't know what they collected. So this is, this is where all the data architectures like the data mesh emerged, and I realized this, sometimes we don't have to talk about the ML ops. So we have to start with the data first, like the you know, the creative base, and so on. So I come from from the ML ops, you know, from the operational side to the Okay, let's first specify, do you actually need machine learning? So like, do you actually what are your use cases? Will the these use case bring a value to your company, so is, you know, all that stuff. So I come to the these ideation phase, where I try to use domain driven design, knowledge discovery methods for data science for machine learning projects, and introduce these kinds of knowledge like domain driven design to data science world and merge them. And then I realized that okay, we have to go more to the beginning to the data. So and this is actually the path data mesh, use cases for machine learning and the ML ops. And

Tim Bourguignon 37:53
I love how you put it going from from from actually measuring machine learning, and then working, navigating back to the problems that you see and saying, Well, we have to, first to clarify this. And first to clarify that. And first to clarify, because it's, it rings two bells. First one is, when I was still doing consultancy, we had exactly that kind of of discussions where we say well, but we want to do this part, the ML part nice ml part. But we actually have to specify the rest, because that's where the problem is, that's where the customers just don't get the bang, bang for the buck right now. It's actually more the data engineering part more than the data science part is, I really have to bring that value first. And if you do your job well, there, and the rest will be making air quotes easy. The rest will be easier, at least and that. And that's what I'm seeing in the company work for right now. Again, we we grew is directly, we have a lot of data. And we've been working backwards our way backward in the recent months, working like Okay, let's go from a data mess with data mesh really start making making sense of all this, maybe the ETL we had is not the right way to do it. So we've been exploring data bricks, for instance. And he can we can start working on this. And now Now we're at a time where we can start working with BI in in these habits, realization, and at some point ml will come but it's really working our way backwards. And then for what again. So I love how you describe this process in the way you work with your customers.

Larysa Visengeriyeva 39:24
Thank you very much but the you just mentioned from data master data mesh, it's a catchy, catchy or you know, like to talk headline, so a cool

Tim Bourguignon 39:40
feature where our CTO picked it up but massaged it as well and say, Well, we have a mess now.

Larysa Visengeriyeva 39:46
Cool, cool. Thank you.

Tim Bourguignon 39:49
So what I hear is or what I see as well but with a big smile on your face when it's okay this and you can see yourself working in this for ages. Is that right?

Larysa Visengeriyeva 40:01
Yeah, we have so many problems. So I can really, you know, I can I can, I can swim in and answered questions, which I absolutely love. So, so I be sure, so I will pick up the next possible problem. And, you know, make people happy by answering this question.

Tim Bourguignon 40:25
And as long as you can scratch your own itch, have a paycheck and find new problems. Everything's fine, isn't it? Absolutely. What we do, awesome. If you had and any advice for people starting with ml ops, they have no idea. They're probably going to go to your website. But where should it go from there? What's what's the first step? If you understood that your problem lies somewhere before the ML? But you don't know more than that? What's the first advice you would give? What is first steps?

Larysa Visengeriyeva 41:01
Oh, you answer the question go to mo minus. Org. So this is actually for, for you know, these websites reflects my journey. So from the beginning from from zero. And, and I got feedback from people who, who read through the website and it's exactly the journey so you can go the end to end process. Look at the sketch, I drove by myself and see all the pieces. Okay. This is this is okay. Okay, this is a shameless. Shameless, but I did my best. I'm sure you

Tim Bourguignon 41:49
did that. And that's fine. I was on the website. Back then, back then, a few minutes before we started recording. That's, that's back then already. I was wondering if there is one place to start more than the others. There's an article about motivation for ml ops, for instance, theories of an article but principles, where would you suggest we should start?

Larysa Visengeriyeva 42:14
Start with designing ml part software? And then and then go to the answer and ml, machine learning workflow lifecycle. Okay. So, and then you will get more like, Okay, I would like to know more. And then

Tim Bourguignon 42:34
that's exactly what I'm searching for this flywheel you have to start and now your finger is gone already.

Larysa Visengeriyeva 42:41
Yes, there's this another one, the Treasurer, which I have for people who are so yeah, so I am maintaining the awesome ml ops repository on GitHub.

Tim Bourguignon 42:58
But we have to add a link to that what would we find there?

Larysa Visengeriyeva 43:02
You will find everything about ml ops, which is written on the web

Tim Bourguignon 43:09
then will either link to that definitely

Larysa Visengeriyeva 43:12
yeah, because okay, it might be overwhelming for like for first the first class can be can be overwhelming, but you you will find there everything. So, the links the references to staff are structured according the topics according to topics within OPSEU or for example, you can find this section about ml ops courses or ml ops communities. And this is my advice for people who are new in ML ops go and find your community and the first community which is really big community and large community is the ML ops community ml ops dot community. So you can you can find, so just join the slack, ask questions, and you will find so many the brand people there too, who will answer your your question who will who will help you in your journey. This is the way to, to navigate in, in any new topic in the tech world or beyond the tech world. So find the community like minded people, join them and enjoy the life

Tim Bourguignon 44:48
couldn't put it better. Like you realize, obviously, oh, I guess you are to be found on this community. Would that be the place where people should contact you if they want To continue the discussion with you or somewhere else,

Larysa Visengeriyeva 45:02
absolutely, you will find me on this community because I'm the early member of this community. And now that we're on so you can find me on LinkedIn. You can find me on Twitter. But don't be so, okay, so. So do there is more personal or technical, personal. And LinkedIn is death serious stuff. Is there. Like the professional stuff

Tim Bourguignon 45:36
there, we have to stay talking with Dr. Liza.

Tim Bourguignon 45:43
So if you contact Larissa on Twitter or slack, you can call her with the first name. Otherwise it's Dr. Okay. Okay, anything else you want to plug in before we call it today?

Larysa Visengeriyeva 46:01
Yes, probably my Festival, the baby. So you're probably the fifth

Tim Bourguignon 46:13
or sixth after the beauty. Oh, god, it's becoming a lot.

Larysa Visengeriyeva 46:19
Growing up. Alright, so the festival. So I am, together with my brilliant team from the inner queue and all the female tech communities, which I'm actually also part of. We're all organizing the woman in data and AI Summer Festival. One day, one night of technical stuff about data engineering, machine learning data science and ml ops. Off topic, festival spaces about about professional development, financial independence, intrapreneurship. And in AI, and the big techno party after all of them.

Tim Bourguignon 47:11
The Techno party went belly up. Absolutely. And it will be three days after this show was released. So if you're somewhere around Berlin, and you hear this when it when it comes out, don't hesitate. Just go there, if not for the tech for the techno party, but I'm sure this will be interesting as well.

Larysa Visengeriyeva 47:38
Probably I will, I will I will notice after they show

Tim Bourguignon 47:42
me some links and I'll add them to the show notes then. It's been fantastic talking to you laughing with you and then listening to the story of going up and downs with PhD or not and and discovering themselves. Thank you very much. Thank you very much for hanging meeting. And this has been another episode of Tempest journey we see each other next week. Thanks a lot for tuning in. I hope you have enjoyed this week's episode. If you like the show, please share rate and review. It helps more listeners discover stories. You can find the links to all the platforms to show appears on on our website, Dev journey dot info, slash subscribe. Creating the show every week. Takes a lot of time, energy, and of course money. Will you please help me? Thank you bringing out those inspiring stories every week by pledging a small monthly donation, you'll find our patreon link at Dev journey dot info slash donate. And finally, don't hesitate to reach out and tell me how this week story is shaping your future. You can find me on Twitter at @timothep ti m OTHEP corporate email info at Dave journey dot info