WillowTree has made pioneering strides in using AI throughout the software development process, and one of our key learning areas has been the utilization of GitHub Copilot. This AI tool facilitates a streamlined and efficient approach to development-related tasks.
In our recent webinar, three of our seasoned software engineers — Nish Tahir, Kris Horn, and Chris Sellek — joined me to share their experiences with GitHub Copilot and demonstrate its potential in optimizing code efficiency during their workflow. Let's dive into the key takeaways:
- Nish Tahir highlighted one significant advantage of GitHub Copilot: its capability to rapidly generate boilerplate code. By enabling developers to focus on core code logic instead of repetitive boilerplate material, devs can significantly increase productivity and code quality.
- According to Kris Horn, GitHub Copilot can drive notable progress in unit test generation, contributing to application robustness. Copilot can offer unit test case recommendations in seconds, integrating seamlessly with developers' usual IDE. This, coupled with the GitHub Copilot chat feature, makes it highly convenient to troubleshoot any coding challenges.
- Chris Sellek often takes on challenging tasks like refactoring legacy codebases and makes the most of Github Copilot when working with unfamiliar programming languages or frameworks. Unfamiliar syntax or coding conventions no longer require painstaking Googling, as GitHub Copilot can significantly speed up the process of learning and adapting to a new programming language.
While these use cases highlighted by our panel of experienced developers indeed demonstrate GitHub Copilot's potential, the tool isn't infallible.
For instance, Chris pointed out that it may sometimes generate code that doesn't adhere to best practices. However, the same could occur with any human developer, emphasizing the need and value of a robust code review process irrespective of toolset.
Similarly, GitHub Copilot can only identify areas with adequate test coverage, but it can guide users toward the appropriate tools that ensure comprehensive code testing.
Finally, understanding when to use GitHub Copilot and Copilot Chat depends on the specific needs of the task at hand. While the former is excellent when you want to get code generated quickly, the latter is better for seeking a comprehensive understanding of what you're working on.
The panel discussion highlighted just a few of the myriad ways in which GitHub Copilot and AI, in general, can streamline and enhance software development. By harnessing the power of these tools, WillowTree continues to revolutionize our approach to creating digital experiences.
Check out the full webinar and reach out to learn more about our Data & AI explorations.
For help getting started with your genAI projects, learn about our eight-week GenAI Jumpstart program and future-proof your company against asymmetric genAI tech innovation with our Fuel iX enterprise AI platform.
Webinar Transcript
Ellie Giles [00:00:15]: Thanks for joining us today as we dig into taking GitHub Copilot to the max. My name is Ellie, VP of Delivery Practices at WillowTree, and I'm going to be your emcee today. Many team members at WillowTree have been early on the GenAI adoption curve, exploring ways to leverage the rapidly evolving tool landscape to benefit how they work and what they deliver. We've seen Copilot emerge as a frontrunner for AI developer tools. With the beta release of Copilot chats giving ChatGPT a run for its money. During our time together today, you can expect to learn from a few different Copilot users at WillowTree about how they get value out of the tool. We'll leave some time for Q&A in case you have follow ups on the tool or specific use cases that we can field while we're together. Before we jump in, let me introduce those Copilot users I referenced so you know who you're hearing from. Nish Tahir is a Principal Software Engineer at WillowTree who will "work on pretty much anything." His words, not mine. Some of his favorite types of problems to solve include compilers and developer infrastructure. He's been the main developer and maintainer of the Kotlin syntax highlighter on GitHub for over five years and outside of work plays a lot of Magic: The Gathering. Kris Horn is a Software Engineer at WillowTree, currently focused on React Native. She has a passion for learning, teaching, accessibility, quality, user experience, and analytics. On the side, she has built and maintained an analytics eCommerce app that sells all her favorite Legend of Zelda games. Outside of work, she's a big crafter and can be found sewing and creating various things. Chris Sellek is a Staff Software Engineer focused on web frontend. His favorite type of problem to solve is a complex refactor to make a legacy codebase more efficient and easier to work in for devs. On the side, he wrote a 100,000-word fantasy novel and is now looking for literary agent representation. Outside of work he enjoys hanging with the family. All right, let's start hearing from our panel. Nish, let's get started with you. I'm sure you could come up with many, but what's one reason that you would advocate for a Copilot as a development tool?
Nish Tahir [00:02:32]: I think in my experience, it really helps take away focus from all of the boilerplate stuff that I would typically have to write day-to-day. And it kind of just lets me focus on the things that I think are kind of most important. And I think usually I would know exactly what it is that I want to put down, and when I'm in sync with Copilot, it kind of just figures it out and really helps improve the speed of my hands so I can kind of just get down all of my thoughts as one very fluid stream of consciousness faster than I would normally be able to type it.
Ellie Giles [00:03:16]: That's awesome. We'll definitely revisit that in a moment. Kris Horn, what's one more thing that you'd add on.
Kris Horn [00:03:20]: With GitHub Copilot I can achieve greater code coverage and unit testing in my application with minimal time commitment, which is super awesome. GitHub Copilot can also recommend unit test cases in a matter of seconds and with the addition of GitHub Copilot Chat, I don't even have to leave my IDE to troubleshoot any errors that might come up along the way.
Ellie Giles [00:03:47]: Nice. Chris Sellek, one more. What would you tack on?
Chris Sellek [00:03:51]: Yeah, it is super helpful for me when I'm working in like a new language or framework that I've never worked in before. You know, previously it would be lots of Googling just to figure out how do I even declare a function in Python, right? Like if I've never worked in Python. Now I'm in a Python file and it knows that I'm in a Python file. I give it a comment and I'm like, "Hey, we're going to write a function." And it's like, "Okay, here you go, Boom." And I just hit tab and there it is. And then especially, I've been doing this a long time, eight years plus. And so, at this point, syntax, I know what I want to do, right? And the syntax is not important. I don't care about the syntax. The syntax is going to change from language to language, but it's all doing the same thing. And so I just want to get the syntax out of the way so I can just write the code that I want to write.
Ellie Giles [00:04:47]: All right, our panel identified the three high impact use cases for Copilot. Let's get into some quick demos to bring this to life a little. Nish, let's start with you again. Why don't you walk us through how you use Copilot for generating boilerplate code?
Nish Tahir [00:05:02]: All right, cool. So for this demo, I've put together a little project. So this is a parser, and it's set up to parse a kind of mini programming language here. And what I wanted to showcase was how Copilot kind of helps in an environment where you would typically be doing things like programming language development, which is something that I enjoy. So when building programming language parsers, it tends to get a little boilerplate-y and repetitive because programming languages tend to be pretty large and very robust. But with the right kind of prompting, Copilot can be really helpful. So in comments here, what I'm adding is kind of my own mini spin on BNF notation. So this is a standard for describing programming languages. And I kind of, you know, modify things here and there just to keep it simple and easy to read. It doubles as great documentation. But Copilot can, based off of the notation that I add in there, can help me generate stuff. So what I can do here is declare a port and you know, just right off the bat, Copilot kind of knows, "Okay, the pattern so far is for every language, for every little snippet of BNF I provide, there is a corresponding function." So it kind of knows that, hey, we actually need that. And following the pattern from some of the other functions, it knows, "okay, we need to build a kind of, you know, little holder here for my data." So I can prompt it and it knows, "hey, we need to kind of fill out some other portions of that little holder there." And then it just kind of takes over. So it's looking at the comment prompt and kind of just filling stuff out. If I give it a second, it knows, "hey, we need to uphold the token, then we need to match it. We need to match the token type that's output from the tokenizer. And then we need to use that to populate the rest of our data there." And then at the end we return the port from the then we return the port that we kind of created over there. So it makes it super easy and really, I'm more-- the thing I generally will really care about is this and Copilot can kind of take over in filling out the rest of the code for me. Makes it super easy to iterate because I can just blow away a function that I don't really need anymore or that I want to adjust and, you know, just retype my little template here and it can do the work of building out the rest of it. So yeah, super easy, super, super fun. And yeah, that's about it.
Ellie Giles [00:08:45]: Now let's go a little deeper and unit test generation. Kris Horn, would you showcase how you approach writing unit tests with Copilot?
Kris Horn [00:08:53]: Yeah. So let's look at an example of how GitHub Copilot and GitHub Copilot Chat can assist in writing and debugging unit tests respectively. On my screen here, you'll see that there are already a few unit tests written out for a log in feature. Let's see how GitHub Copilot can help us finish out the remaining tests for a password validator. So I'm just going to start typing out here and already GitHub Copilot understands what I want to do and has already made a suggestion for the describe block. It actually has started to generate some prompts for all of my test cases. Normally you'll see like a little block at a time, but I like these and these are the things that I actually need to test. So are they accurate? Are they any good? Let's find out. Yarn test. Oh, well, we have a failing test, but it doesn't look like it's in my helpers file here. So let's check out this test that's failing. We have multiple elements in the login screen. So I navigate over here to my log in screen test. I am testing out whether or not I can log in, click that log in thing. And it looks like I do have two instances where there's text for login. Now how do I fix this? I have a few ideas myself. However, GitHub Copilot Chat can help me with this. It actually starts off by asking me, prompting me with some questions that I might ask. So that's really useful and helpful to begin with. But I want to ask something kind of specific, for this test for this case that I'm trying to solve. So I've already sort of generated a prompt down here to help me understand what I want to do. I gave it some context. It's really important to provide your chatbot with a little bit of context and let's see what it suggests. So boom, real quick. Just like that. I have a suggestion: getByTestIds. Hmm. But for me and my team, getByTestId doesn't really match our testing philosophy. So is there maybe another way that I could test this? Let's find out. So, "is there another way to fix the above error without using TestIds?" Oh, I like this response a lot better. So it prompts me to use getByRole. And this is a little difficult to see with a squished up setting here, but I can see right here. In my tests that we viewed briefly before I had this login button that I was getting ByText, which was the issue. And now instead of getByText, I want to getByRole. And this will target the role of that button and hopefully pass our test. So let's see how it works. Yarn test. And there you have it. We have quite a good amount of test coverage here. Looks like I could always add some more for this authentication hook that I'm running. But overall, GitHub Copilot helped me to generate really quickly some unit tests on the fly. And GitHub Copilot Chat helped me to debug an issue that I was having in a unit test that was already existing and it made it better and more valuable for the team.
Ellie Giles [00:12:51]: Last but certainly not least, Chris Sellek. Bring it home and show us how you take advantage of Copilot to ramp up on a new language or framework.
Chris Sellek [00:13:00]: All right. Thanks, Ellie. So, yeah, I've been working in TypeScript, JavaScript for a long time now, and that's just so that's the language I'm the most comfortable with. But you can see I have here a quick function that says is a string, a palindrome, right? It takes a string and a palindrome as a string that's the same forward and backwards. It takes a string. It compares it with, it splits it up into an array because you can reverse arrays and then it joins it back together into a string. Is it equal to the same, to the original string it's doing that. Now, onto the demo portion here: I know almost nothing about Python syntax. I've never worked with it, very little bit here and there, and that was a long time ago. So I would not know the first thing. I don't even know how to declare a function in Python, but let's see if GitHub Copilot can help me just do that here. So I want to write a comment to tell Copilot what I'm preparing to do to kind of like push it in the right direction. But I actually, I don't even know what the syntax is for comments in Python. Thankfully, I have my IDE here, which is another tool that I have access to and I know the hotkey for turning in-line into a comment, so I'll do that now. All right. Pound sign apparently. Cool. Okay. So I'm going to say "determine if a string is a palindrome" it picked up on that, which is cool. There we go. Okay, so now already cool. I see this is how you declare a function. Awesome. And then, oh look at that. Okay. It just did it for me. And so now we can maybe test it. Yep, let's do that. And then down here, if I run this command again, we should see True. Awesome. And then if I give it an S there, it's not going to be the same forwards and backwards. So now it should be False. Cool. All right. So immediately I can see several things: again, as a more senior dev, all of the patterns going on under the covers are all exactly the same. I just don't know the syntax, right? And I don't want to have to Google around to figure out, "well, you have to do 'def is palindrome' and then a colon and blah blah blah." I don't have to worry about any of that. Copilot just took care of all of that for me. And so I see this is how you declare a function. Apparently return with the keyword return. They use return as well. Cool. I actually have no idea what this is doing, but that is where Copilot Chat comes in. So I can say "What's this syntax doing" and then give that to it there. "The syntax is used to reverse a string in Python." There we go. And then it even goes further to explain why. "Start at the end of the string, negative one, and move backwards by one step, negative one, until the beginning of the string is reached." Cool. So awesome. It's all right here. It's all my IDE. It's all immediate. I don't have to go stumble through several different Google searches, Googling one keyword, reading a few articles, Googling the next keyword reading a few more. I don't have to Google any of that, it's all right here. It's all immediate and it's all right in my IDE, which is super awesome and super quick.
Ellie Giles [00:16:15]: All right. Thanks, Chris. Thanks, everyone, for walking us through those use cases. It was helpful to see that come to life. With that, we're going to spend the rest of the time on Q&A. So I'm going to do is try to select an assortment of questions that you all can field related to your demos. First up, I think this is probably most relevant to you, Chris Sellek. I see a question, "Especially if you're using it to write code in a language you're less familiar with, what's the risk of Copilot suggesting code that doesn't follow best practices?"
Chris Sellek [00:16:52]: Hmm. So, yeah, Copilot will definitely spit out code that doesn't follow best practices. It just depends. You know, it's just the way that language, large language models work. It's whatever statistically is most likely to come next kind of thing. And if most people have bad software practices in the specific thing you're writing, then yeah, but that kind of stuff, I think that's always a risk. I don't think that that is. So like as a senior dev, I would see it and probably very quickly, even if it's in a language I'm not familiar with, I'd see that issue because the security issues tend to be more what's the word pattern-oriented versus like syntax-oriented. And so you can probably very quickly see, okay, we're feeding user inputs straight into the DB. We don't want to do that kind of thing, whatever. And then for more junior engineers, if I don't think that... how do we learn what is best practice, right? We make those mistakes or we see other people make the mistakes and then, you know, a more senior engineer spots it in the PR or something, mentions it in a comment like, "Hey, here's why we don't want to do this." And then that that anti-pattern or whatever is caught and fixed. And so yeah, it's going to have those in there but I think that anyone, any human is going to make those mistakes, too. And then, you know, that's why we have PRs and stuff like that.
Ellie Giles [00:18:29]: Thanks, Chris. That was helpful. Nish, I think this next question most relates to your demo. "As long as you use industry-standard notation, should Copilot be able to act on that?"
Nish Tahir [00:18:43]: Um, yeah. So I think the underlying question here is: what sort of notation is Copilot most able to act on? And I think really what Copilot is good at is pattern matching. And as long as there is a consistent pattern that is being represented, that you're using to prompt Copilot, it will be able to act on that. So I picked that specific notation because it's an industry-standard in the sort of field of kind of language development that everyone's familiar with. And so it kind of doubles as good documentation for me. However, you could in theory, kind of make up your own thing, but as long as you're consistent in the codebase, Copilot will be able to figure it out.
Ellie Giles [00:19:36]: Got it. Kris, let's move on to you. This question I think, is directly relevant to the demo that you walked us through. "Can you use Copilot to identify areas of a project that don't have good test coverage?"
Kris Horn [00:19:53]: So the short answer here is No. GitHub Copilot cannot identify areas of a project that are in need of better test coverage, as an AI tool. However, GitHub Copilot Chat can help guide you to the right tools to understand coverage and better assist in generating those test cases for those components.
Ellie Giles [00:20:17]: Got it. That directly leads us into one more question that I'm going to tack on. "Can someone talk about uses for Copilot versus Copilot Chat?"
Kris Horn [00:20:29]: Yeah. So I think the usefulness about Copilot is if you're in your code and you're starting to type and you want code to be generated that you sort of have an idea for. That's a good spot for GitHub Copilot. If you're curious about something, you're not quite sure, How does this work? GitHub Copilot Chat is a really great tool to use just off to the side of your IDE. And then you're like, "Hey, let me just ask you some conversational information here and get some context on what I'm working on," and then it helps to better understand what you're getting from Copilot in general.
Ellie Giles [00:21:07]: Got it. Let's add in one more perspective on that question. Nish, what's your take on uses for Copilot versus Copilot Chat?
Nish Tahir [00:21:15]: Yeah, I like to think of it as like. Fill in the middle versus having a kind of well thought out conversation. So in my use case, or at least like how I use it, would typically be I'd reach to Copilot Chat if I wanted to sort of explore an idea and really kind of talk like kind of rubber duck, talk through like the pros and cons and sort of get a second perspective on something that I'm thinking about. And so it could be something like, you know, I have an idea for a database schema and I want to kind of talk through like some alternatives. And when I'm ready to actually you know, put pen to paper and actually write it out is where I'd reach to Copilot to help with that.
Ellie Giles [00:22:15]: OK. Thanks all for running through those questions. At this point in time, I think we did end up running out of time to answer all of the submitted questions, so we'll look to follow up on those offline and move us in to wrapping up. Before we wrap, I'll finish by highlighting that at WillowTree we're seeing that Copilot used to its fullest potential is especially valuable in achieving a number of outcomes. We touched on some of them today, but certainly not all. We encourage you to check out some of our other content on the subject and let us know through the audience survey whether you have interest in us covering different aspects of the topic. Thanks for joining and have a great day.
