Send us a Text Message. (https://www.buzzsprout.com/twilio/text_messages/1326658/open_sms)
JupyterCon 2023, the conference on all things Jupyter was held in Paris between 10-12 May 2023, followed by 2 days of hands-on “sprints”.
Jupyter is a very popular open source platform with tools such as Jupyter notebook/lab and driven by a very active community. There were a number of excellent talks from a range of different subjects. I had the pleasure to meet and talk to a number of people, see the interview list below.
Order of Interviews:
• Leah Silen and Arliss Collins from Numfocus 02:04
• Franklin Koch (MyST) from Curvenote 04:59
• Nicolas Thiery (Paris-Saclay) 09:13
• Sarah Gibson (2i2c) 13:19
• Ana Ruvalcaba (Jupyter Executive Council) 18:57
• Fernando Perez (Jupyter Executive Council) 23:48
• Raniere de Silva (Gesis) 29:56
Links
• https://jupyter.org Jupyter project
• https://jupyter.org/enhancement-proposals/79-notebook-v7/notebook-v7.html# Release notes for the new Jupyter Notebook v7
• https://jupyterlab.readthedocs.io/en/latest/getting_started/changelog.html#v4-0 Release notes for JupyterLab v4.0 (further incremental updates of v4 are available)
• https://www.youtube.com/@JupyterCon YouTube channel for JupyterCon 2023
• https://cfp.jupytercon.com/2023/schedule/ JupyterCon 2023 schedule
• https://www.outreachy.org Outreachy project
• https://numfocus.org Numfocus project
• https://data.agu.org/notebooks-now/ Notebooks Now initiative
• https://myst-tools.org MyST tool for scientific and technical communication
Upcoming RSE conferences:
• https://rsecon23.society-rse.org UK RSE conference in Swansea 5-8 Sep 2023
• https://hidden-ref.org/festival-of-hidden-ref/ Hidden Ref in Bristol, UK, 21 Sep 2023
• https://un-derse23.sciencesconf.org Unconference of the German RSE society deRSE in Jena 26-28 Sep
• https://us-rse.org/usrse23/ 1st face to face
Support the Show. (https://www.patreon.com/codeforthought)
Thank you for listening and your ongoing support. It means the world to us!
Support the show on Patreon https://www.patreon.com/codeforthought
Get in touch:
• Email mailto:code4thought@proton.me
• UK RSE Slack (ukrse.slack.com): @code4thought or @piddie
• US RSE Slack (usrse.slack.com): @Peter Schmidt
• Mastadon: https://fosstodon.org/@code4thought or @code4thought@fosstodon.org
• LinkedIn: https://www.linkedin.com/in/pweschmidt/ (personal Profile)
• LinkedIn: https://www.linkedin.com/company/codeforthought/ (Code for Thought Profile)
This podcast is licensed under the Creative Commons Licence: https://creativecommons.org/licenses/by-sa/4.0/
[Music] welcome to another episode of Code for thought and this episode takes us to the city of light Paris which hosted this year’s that is 2023 Jupiter con the conference for all things around the Jupiter platform over 3 days between the 10th and the 12th of May 2023 developers entrepreneurs volunteers researchers and students met at the city de lasance in the northeastern District of Paris this has been the first face-to-face Jupiter conference since the pandemic the first time it was held in Paris and the first Jupiter con for me personally so there was quite a bit to be excited about and in this episode I want to give you Impressions from the conference and let you hear from some of the people I met including members from the Jupiter executive Council Jupiter con was organized by a strong team led by sane Kay from quantac Gail Allington who is Jupiter’s community events manager and N varo who is a researcher at the French national Research Center CES not to mention of course many of the volunteers who dedicated their time and energy to help run this event smoothly and you’ll hear from one of them rier a little bit later Jupiter itself is a nonprofit and open- source project it’s known for the popular Jupiter notebooks and Jupiter lab apps which are used by millions of people worldwide Jupiter success is built on a strong community of contributors be that individuals and volunteers or organizations and companies it also of course relies on strong help from sponsors and partners partners like for instance num Focus so I’d like you to meet the executive director Leah silen and the developer Advocate Alis Collins from num Focus who I talked to during one of the conference breaks hello and here I am with Aly and Leah from num focus and num focus is of course an organization that we will find out in just about a minute so can I hand over to you Leah and explain a little bit about what num focus is for the listeners who may not be familiar with it yes hi thanks Peter as Peter said we are with them Focus so um my name is Leah silin and I’m the executive director and sitting next to me is Arles Collins our developer Advocate and we are sitting here with a table full of stickers that are showing all of the numfocus projects that we sponsor that are under our umbrella our legal umbrella and that we provide project services for so numb focus is here our mission is to support open- Source scientific computing projects and we have just little under 60 projects under um under our umbrella that we provide services for both to the internal contributor Community as well as to the user community Through educational initiatives through events such as Jupiter con we also have many other project events that we help out with as well as um the pi dat event series and our works very closely with our projects and Mak sure that we’re on track with those Services AR you want to say a few words sure absolutely so as the developer Advocate at num Focus I um as Leah said we have 60 projects so my day-to-day job is to check in with each of these projects see how things are going what are there what are they working on what are they passionate about at that time and then figuring out how num Focus can provide services that will help them achieve the goals that they’re looking for the conference itself was organized in different parallel tracks as is usual these ranged from education Community tools and practices to data science and Enterprise infrastructure tutorial and practical sessions were also available and if that wasn’t enough the conference was followed by two days of Hands-On workshops right after svean Kay kicked off the conference on the 10th of May there were a range of interesting talks one of which touched on the subject of scientific publishing this is a subject close to my heart Franklin C from curve nodes presented a tool called Mist which stands for marketly structured text that is how to use markdown in its extensions in Jupiter notebooks to produce interactive and compelling scientific papers for communication and luckily I had a chance to have a chat with Franklin about this after his talk hello Franklin nice to meet you you gave a very interesting talk about Mist perhaps you can quickly introduce yourself thanks yeah I’m uh Franklin cook I work for a company called Curve note um and we also do a lot of uh work with executable books which is what mist is part of that’s exactly what I want to talk about because we’re talking about workflow and scientific papers how do you see Mist fitting into that mist is trying to the big pictures that is trying to improve scientific public application and enable authors who have interesting computations in their work enable them to publish their science in a way that respects that interactivity so mist is a few different things I guess it’s like it’s sort of a markdown format that you can use to write your paper that is an extension of common Mark which is a basic markdown format and it like takes uh inspiration from restructured text and adds a few features just to make it nice for people who already work in jupyter notebooks so yeah there’s that format to author in and then Mist also has tools to take that content and build websites that maintain the interactivity from the Jupiter notebooks that’s good because I mean I saw a presentation earlier today and that was really great to see the kind of interactivity which goes beyond the standard PDF and yet journals still love PDFs how do you get from your missed publication and website to a journal yeah so I mean I think that this starting a movement kind of thing like a yeah changing how the publishing industry works is not going to happen overnight as I think probably anyone would say but where curve note is starting is we found a few Publishers who are willing to kind of take take the plunge and do things differently and most of these right now are in pre-print stage so we’re thinking about how peer reviews work and how we get further in the formalization of the process but right now there are a few journals who are just like oh yeah we’d rather have a nice website where you can click around and run code rather than just look at static PDFs could you name the journals yeah I mean I guess physiome is the the first one that we released and is marketing and then we’re working with notebooks now which is an hu Journal they already see notebooks as first class medium for scientists writing in and want to make those notebooks become be publishable as like actual Publications and so we’re with them kind of to figure out what that looks like and where we need to formalize things to align with the publishing industry versus where we can kind of flex our muscles a little bit and do more interesting things the need for a change in scientific communication was a subject that came up several times during this conference and not laced in the keynote from Alysa Goodman from Harvard University on the second day of the conference in her talk she demonstrated a tool called glue and glupiter to help visualize astrophysical phenomena in 3D which was all very fascinating but back to the subject of scientific communication because everybody recognizes that changing the world of scientific communication will take time but it is encouraging to see as Franklin said that some Publishers have come on board and maybe we will be able to say goodbye to the PDF at some stage in the future education is another area where Jupiter shines as I mentioned an entire track was DED at to this conference an increasing number of schools and universities are using Jupiter notebooks for instance so much so that the question becomes how to provide an infrastructure which can help providing jbit in the classroom more effectively listeners of this Podcast May remember my conversation on the subject with Nicholas tii from Paris saay and his project proposal Candice being a distinguished contributor to Jupiter it was no surprise to see Nicholas at this conference and and I was lucky to grab a few minutes of his time for a chat and just before we met we heard from the project Capital which is also working to improve infrastructure with Jupiter in schools so here now my conversation with Nicholas hello and uh with me here today is Nicola tii and listeners may remember Nicolas from our interview about Jupiter in the classroom so I guess this is one of the reasons why you’re here today hey Peter it’s good to see you again we also heard a talk about Capital yesterday from another one who is working in Jupiter in the classroom and I wonder if there’s any relation between that and candies yes definitely actually capital is one of the many projects that inspired Candis in the first place so essentially what happened the last years is in many different context so in for Capital it’s in for High School teaching also for many universities we have had the need for infrastructure Jupiter infrastructure and many people in each little institution built this kind of in infrastructure capitally what they did to support the needs for high school and the goal of Candis is inspired by all those little needs to try to see could we get together and have instead of each spending energy on one side to build one piece of infrastructure or try to do things as common as possible to solve all common needs and when I looked at the slides yesterday there were an enormous amount of people actually registered users on Capital I think he were mentioning 280,000 or more and so you would think there is actually a pressing need for some common infrastructure so what is the difficulty in convincing the authorities to give funding well the main difficulty is that yes there are pressing needs right now but currently the state is also having to face or other big pressing needs like shall we buy munition for Ukraine shall we invest into hydrogen energy and and things like this so this is really at at at the stage really this kind of political decisions that are needs to be taken and I must say I’m glad not to have to take this kind of decision yeah I’m glad too so I’m glad not to be a politician Jupiter in the classroom I think at least in France this seems to be adopted very well are you working with other European partners or other partners in the world on this definitely the goal of CIS is to make contact with other places that would have similar projects and the long run to not have one thing for friends but one thing for everywhere there is this big question is whether we should have one infrastructure that fits everybody or should we take the path of making it easy to deploy such infr infrastructure so that if a country needs one or if an institution needs one it’s easy for them to to deploy so we’re kind of exploring this dual way of deploying something at scale and making it easy for overs to deploy it and if over have similar needs and if there is opportunity to join forces yes definitely we should do it and actually this is already what we are naturally doing it in the jupyter ecosystem because everything is done in a quite modular way and we’re focusing on building bricks so that we can easily collaborate on these bricks and then maybe the platforms will be different but as long as 99% of the bricks are common it’s not so much work to have own B book infrastructure for spe specific needs let’s move from the education track to the community tools and practices track where I met with Sarah Gibson actually for the first time face to face Sarah gave a number of presentations at the conference but I want to highlight one of them which was on the subject of outreachy outreachy is an organization that provides internships to people from underrepresented groups and to experence experience of stomach bias which is a noble undertaking but how do you actually organize this in practice and what lessons can be learned and this was the subject of my conversation with Sarah unfortunately the only reasonably quiet place we could find was close to an air conditioning tuck so there will be a little bit of humming background noise in the recording hello Sarah again Sarah you’re regular visitor and guest on this podcast and I’m very grateful for that hello it’s wonderful to be talking to you again and this time in person how amazing it is and there is a photo as approv yeah there is anyway I want to talk to you because you’ve been given two talks yesterday yeah two talks yesterday so it’s been a visual conference for me indeed because I thought it was great the outreachy program maybe you can quickly explain a little bit to the listeners what that is and then we can go into the questions yeah so Outreach is a great nonprofit organization that puts interns into open source and open science programs and projects in order to learn about how to do open science and open source and work on some really cool code bases and they primarily offer these internships to people who are from under represented groups particularly in the tech system these internships are paid they’re fully remote and they last for three months and they happen twice a year I mean there are lots of stuff to celebrate because he got an internship which is paid and for 3 months but there were some challenges and I think that’s something that I would like to talk to you about I think one of which was onboarding and maybe you can explain a bit more about that yeah so I said in my talk like one way to figure out how good your onboarding documentation is is to have like 60 plus interns turn up at the same time and try to like do stuff so it was a challenge to manage the influx of questions how do I do this what do I do next what’s the next thing I don’t understand that and sort of making sure that that was a single Communication channel that was clear for the interns and for everybody who was trying to help as well we learned a lot about our onboarding materials and were able to improve those as a result like definitely a stress test emphasis on the stress but um we got some really good results out of it indeed and I think one of the questions that I have in particular is the first issues so you have a newcomer you know showing up on Monday bright and early at 9:00 in the morning and what do you give them to do how much work does need to go into it to make that work it’s a lot so the new first issues I was specifically talking about were for the contribution period which is like instead of an interview so they’ve not like agreed to take the projects with us yet and similarly we’ve not agreed to hire them it’s a really fine balance between giving an issue that is is small enough and well scarped enough and achievable enough that someone who may have never worked on GitHub before can like start working but you also don’t want to make it so easy that everybody piles onto it and everybody’s doing it and they do it multiple times and everything and I feel like writing a good first issue in any context is really hard cuz like you have to think about exactly what the problem is what the scope of it is what piece all need to be touched and able to actually provide a fix and like what does an appropriate fix look like so that someone who has no context can get that context and start running with it compared to as you get experienced in the project you probably know some of the things of like they want to do this and I know that doing that kind of thing involves this piece of there and like there’s a post request there or something so I know that we can be able to do this that kind of context of the project only comes over time it’s also a thing that’s really difficult when you have the context it’s really difficult to be aware of that and acknowledge it and like actually write it down in a way that another human being without context doesn’t understand so it’s a lot of effort and then you have to do that a lot of times and you also have to specifically with these first issues for outreach e you want the result that the intent the applicant gives you to like tell you something about their ability to complete the larger project that they’ll be paid to do as well yeah it’s a multi-dimensional optimizing scarping problem and it takes a lot of effort well we talked about the education and the community track but what about Jupiter itself we mustn’t forget that core Jupiter services and applications themselves are constantly being updated and so the Jupiter team presented new versions of Jupiter lab version 4 as well as a new version of the popular Jupiter notebook version 7 which at the time of recording is in beta the latest version of Jupiter lab has a new text editor a new extension manager and improved document search likewise version s of Jupiter notebooks will see a number of features introduced that come from development of Jupiter lab there are links in the notes and I encourage you to check them out give the new versions that try Technical Innovations are not the only things though that are changing at Jupiter as the number of users and contributors grow the organization needs to make sure it can do justice to all the different needs and innovations that are happening within the community and so on the second day the executive Council of Jupiter presented a new governance model which was brought to life in December 2022 one of the members of the executive council is Anna ruval cover and I was able to talk to her about these changes briefly thank you very much for taking the time and we’re going to talk about Community but perhaps you can start talking a bit about your background and what it is you’re doing at Jupiter yes so I am the only nons software engineer on the project Jupiter executive Council I can share a little bit about my background for the past seven years I’ve been Grant funded to work on things like events operations Human Resources marketing all of these noncode contributions that I in particular am interested in and able to make Jupiter is quite interesting so you have roles that are actually paid and you have lots of volunteers so how do you keep all of this together that is one of the challenges that we are actively trying to solve right now I don’t think we have the answer but we definitely have the next step and the next phase in our project trying to solve for this there are a lot of competing interests at times when it comes to something like this there is industry involvement there is academic involvement and there is volunteer involvement lots of different types of amount of time that people can give and right now the best model that we can come up with is what we’re essentially testing right now right so there’s an executive Council that manages all of Jupiter’s interest um across the 11 different sub projects there is a software steering Council which is made up of one representative from each of the sub projects the software steering council is tasked with making important technical decisions for the project previously our steering Council was responsible for both things so we’ve separated out two different leadership bodies we also have a group of committees that will report to the executive Council who are also tasked with a number of activities like the diversity equity and inclusion committee there is the community building committee we are just in the STA early stages of a media relations committee we are needing to form a code of conduct committee to respond to any complaints or investigations that need to be done so essentially we’ve taken a lot of different responsibilities that were concentrated in one single steering Council and spread them out among various committees and named them as opposed to just having them be floating or unclear you have a large number of volunteers uh contributing to the whole thing in companies or in organization it’s relatively easy there is sort of kind of an instruction this is what our policies are and then people are following that’s because they’re contractually obliged in theory with volunteers with an army of volunteers it’s a little bit harder isn’t it so how do you tackle that I think we do it through establishing a common interest or a common goal for example I lead the diversity equity and inclusion committee but none of those people report to me right so we meet on a weekly basis we decide what is important to the group and we volunte here to take on different parts of what needs to be done essentially what it comes down to is trying to find consensus and then we do have a decision-making guide that helps us figure out what to do when there isn’t consensus so it’s a combination of defining who has the power to make the decision and then tasking that group of people with finding a way to move forward but yes you’re right it is challenging because there is no manager who is telling all the people what to do somebody might have 5 hours to give this week and next week zero it’s not a consistent path forward but somehow we keep moving down the path and moving forward another member of the executive council is Fernando Perez he’s also the co-founder of project Jupiter and the 2 i2c initiative in her talk on the second day of the conference Fernanda reflected on two decades of Jupiter and presented his thoughts on future strategy and the Community he mentioned one of the classics in business literature the innovators dilemma by Christiansen this book describes the tension between staying Innovative and growing an organization and a business in the following conversation Fernando explains why he mentioned Christiansen a little bit further in my talk I kind of left the final slide with a cover of the innovators dilemma book I do think it’s something that the Jer Community needs to think think about a little bit because we are a project that even though we’ve grown with relatively limited resources and small teams compared to say the the sizes of teams that industry uh can unleash on projects right a lot of this was built by a handful of people at the beginning and and now there are there’s a larger distributed Community but the people who can really spend a lot of their time officially working on the core of Jupiter is is limited and yet we support in a sense indirectly through the kind of multiplicative power of the internet and open source a user base of probably many many millions of people who use Jupiter across the world we know from data that has been released for example by GitHub at this conference that there are millions of notebooks on GitHub being used we know all the tech basically all the major tech companies in the world have products that are based on Jupiter I teach students at Berkeley courses with thousands of students our university has hubs that teach tens of thousands of students uh on campus and that’s true across other institutions in the US and abroad how do we maintain that level of a user base that needs things to evolve in a very predictable way that may not have the resources to be upgrading to the latest and greatest that are is not spending their time tracking what’s happening on GitHub while simultaneously being Innovative because we want to innovate we want to create new things well the world where I python started 20 years ago and then the world where Jupiter kind of became what it is today say 10 years ago is not the world we are in today regarding questions like open science and the role of AI and the role of the cloud and the role of these tools in everyday education industry and research how do we maintain that balance right between stability and Innovation is very much the innovators dilemma that the book described and I think us as a community should be mindful of that I it’s a problem that doesn’t have easy answers big organizations have cracked their teeth on this uh I’m not claiming to have answers but at least I wanted to flag for audience that neither end of the spectrum of let’s focus only on the shiny nor let’s just make sure that everything keeps working the way it is is going to be a satisfactory answer and but maintaining that dynamic equilibrium between those two end points is not an easy task and at least I wanted to kind of bring it up and surface it for our community to reflect on it we’ve seen a number of talks now actually at the conference mentioning we need to replace the PDF we need to replace this how do you deal with all these kind of pressures well we are working on actually contributing to replacing the PDF in some fairly specific ways a number of people who are at this conference myself included for example are involved with an initiative called notebooks now from the American geophysical Union so for reference the American geophysical Union is one of the largest scientific Societies in the world it hosts an annual conference um in December every year in the United States that brings in tens of thousands of scientists they publish virtually the vast majority of the us-based large scientific journals across earth planetary and space sciences that is not astronomy but it does include a lot of NASA work that has to do with planetary sciences and remote sensing of Earth and then everything in Earth from chemistry to biology to environmental issues and so that is a humongous scientific context it’s also a scientific context that is highly relevant because that’s where all kind of climate science happens at agu their publisher is uh Wild so agu is a scientific Society their journals are physically published by Wy which is one of the largest scientific Publishers in the world and the Alfred PE loan Foundation who were the ones who originally funded the first grant to create what was at the time uh known or to to really grow the IPython notebook we had a prototype but then they funded really what allowed the IPython notebook project to grow back in 2012 um that same program at the Alfred pone Foundation is now funding agu together with their publisher and together with the open source Community to build a prototype of what would it look like to put notebooks in the actual publishing pipeline right not as a little add-on not as turn a notebook into a PDF and put it in your supplementary materials not as in well leave a link somewhere in your paper to a binder and hope that people will click on it no how do we make notebooks a first class citizen of the formal peer-reviewed publishing pipeline of a major scientific piece of the scientific landscape we’re working on that it’s a public project there’s information about it at agu notebooks now and uh the idea is to produce a working prototype later this year the people at this conference there have been meetings about the lowlevel gr details of the PDF and XML formats and all that we’re working on it because we do actually want to try to innovate I mean I was I was bringing up the innovators dilemma as a challenge for us because we do have this tension between creativity and a large user base in in some ways but we are very much trying to create new things and push in these spaces that work on Miss on Mist for example is precisely thinking about publishing in the future communicating computational narratives in the future for everyone not just for computer scientists or physicists or mathematicians but for everyone how do we build things that are friendly and interactive that you can teach someone with no Computing experience with but that you can use to share the most advanced research in the world and I’ve seen a lot of work in at this conference from the community in that regard towards the end of this report of this conference I’d like you to meet rier like Sarah I met rier only online so far he works for a research center in Germany called Jesus it was great to meet him finally in person and I was really impressed in which Vanier rolled up his sleeves and stepped in to help out as a volunteer so for instance by chairing a few sessions volunteers like him are the backbone to community events like this and I hope he will inspire some of you to volunteer for future events hello rier it’s great to see you here at Jupiter con how has the conference gone for you so far it had been a pleasure to me to you again Peter the conference had been great is a lovely opportunity to meet old friends and make new friends there is a couple of things that I was really impressed with one of them is the number of talks about our accessibility which unfortunately is a common issue that saw in lots of Open Source projects which it get potentialized by this vicious cycle where people that blind or have other accessibility issues they end up not contributing because in the first place they not are uses because the software is not accessible so there’s very interesting talks different projects that trying to change that and really looking forward to the next conference you see progress on this area but also on all all the different projects that people had been showing cases yeah I agree it was absolutely impressive this conference and very energizing and you’ve been volunteering to chair some sessions I saw oh yeah like as a community conference there’s lots of volunteering task that is needed one of the organizers just emailed me before uh one day before the conference saying hey we you have some chair slots available can you volunteer so I just step in and I recommend everyone to be uh able to volunteer as well it’s a great opportunity to meet some of the speakers and meet some of the other attendees highly recommended anyone to step in and be part of more active part of the conference so if you’d like to help out the future Jupiter cons or other conferences for that matter don’t be shy raise your hand and help out so this has been really an eventful 3 days energizing and exhausting for me at the same time I’m sure other feel the same there were really excellent talks at this conference and you will see them on the Jupiter con YouTube channel which I will share in the episode notes apart from the talks conferences are also a place to meet friends and meet new people and this conference offered this in Spades and I can’t thank the organizers the many volunteers and anyone who made this happen enough too often we take these events for granted but they are a gift really and can only happened with a hard work of dedicated people and so I look forward to another great Jupiter con hopefully next year and while beond the subject of conferences allow me to shout out a number of events coming in Autumn 2023 starting with the UK RSC conference in swans between the 5th and the 8th of September on the 21st of September we can look forward to a one-day event on hidden ref in the City of Bristol in the UK Germany will hold its un conference in y between the 26th and the 28th of September and the US will hold its first RSE conference ever in the city of Chicago between the 16th and the 18th of October and this will be a very H conference Autumn indeed as per usual you’ll find the links to these events in the episode notes oh time’s up see you next time but before I forget this podcast is covered by the creative common license see you