Industry Mentor (MITPOSSE) Overview

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: Welcome, everybody. So this is the one group meeting that we'll have with all you folks. We really appreciate your willingness to come and help us teach 6-172. Saman will explain what the course is all about, and then I will give a presentation, a little bit, of what your expected responsibilities are, and then Eric, who is one of our TAs, will present what the first project that they've been working on is all about.

We think we have a pretty good acronym, OK, MIT POSSE. Masters in the Practice of Software Systems Engineering, OK, and it is very much like a posse, in that we're trying to sort of deputizing you folks to help us deal with this mass of students that we deal with. There's no way, given the resources that we have available at MIT, that we could possibly give the attention to each of the students examples of code, and so our strategy is to go out and deputize some people to help us teach. As we'll discuss your responsibility is not to do any grading, OK, or even any understanding of the course content, OK? But rather to ask good questions of the students so that they understand what it's like to do software development, and you'll see that most of the students are very immature when it comes to coding practices.

They have taken a minimum of one systems course before this, basically.

SAMAN AMARASINGHE: Probably two classes where they write very small programs--

PROFESSOR: Where they write a very small programs, so they have no sort of sense of writing scale. Moreover the class that we're teaching because it's on the topic of performance engineering, they are inclined to break all the rules in order to get the best performance. And that's part of our game but of course what we want is, we want the countervailing-- you still have to make things maintainable, you really should do it this way, yeah it's fine to violate some abstractions here and there, but then you should document that you're violating those abstractions and so forth. And that's what we're counting on you for, is to insert some experience and common sense. So, Saman, take it away.

SAMAN AMARASINGHE: OK, good, so to add to what Charles said, when I was an undergrad, when I took something like a science class, or a physics class, or engineering class, most of the problems had kind of right answer and a wrong answer and then it's very easy to grade those things. But when I took something like an English class, all the papers I wrote, somebody really went through carefully and graded it and gave me lot of insightful comments so I can kind of develop my own style of writing. And what happens in software is the programming there's a style that each individual has to develop their style, but they can't develop a very haphazard style so there's a certain set of good styles, certain bad styles. And the problem that is in a large class we can grade for correctness very easily, we can also look at the programs and have a very fast thing, okay, saying, here are no comments, or your comment doesn't seem to be, it's too verbose, or some very high level feedback, but it's very hard to help somebody build their style.

And so what you guys are coming into is to help us get these young students create their own style of programming. The nice thing about this class-- and also sometimes last year the masters found it a little bit hard was, we are giving them small amount of code. They are doing performance engineering on a maximum couple of hundred lines of code. There's no very complicated class structures or something like that.

So we are not looking at the style and software engineering in the macro level, how to structure large classes but more at the micro level. In each method do they have right style? Are they is it clean? Are they commenting them properly, or can they do it better?

So at that level at least the building blocks that they're-- as they go build a larger and larger program they can hopefully use. So that's what we're looking at, and it helps to instead of dumping you ten thousand lines of code, you only get a couple of hundred lines of code, so you can actually go through it a lot more carefully, and help them out. So this is starting with a crazy eye chart that is our new curriculum that we introduced in the department.

So what you have is some early courses in here, these are basically very general math courses, early courses that do-- they probably write about ten lines of code type thing in these courses, and then we feed into these foundation courses and then after that we sit at, this says 197, actually what should be 172. We sit at that level, so we are sitting at that level, and then the prerequisite for people who are coming is that they're taking a computer architecture course so they know a little bit about architecture. They have taken this introductory software course, so they have written some Java programs, and so they know a little bit about software engineering from that course, and they have taken algorithms course, so they understand algorithms somewhat, so this is the background of the people are coming in.

At the low end there might be some people who have only written couple of thousand lines of code in their lives, and there are other people who are, written half probably-- the Linux kernel. So there are kind of very diversified group of people. But there's some people who it's really new to them, and they don't even understand why you had to do good nice programming, why do you have to document these things. I mean, for them it's a very new concept, because they haven't seen that. So OK, if you're writing ten lines, OK, why do you need document that, I know how to do this.

So part of that is this is, for a lot of them it's eye opening, and in last year also we talked to students, they actually gained a lot from talking to you guys. It's easy for us to tell them do it, a lot of them say OK, just to get the grade, I will do it, but after that I'm never going to do these things, and they just go off in very like, mad at us for asking them to do it, but when you guys say this is what happens in industry, there's a lot more impact on what they're going to do.

And so I would go through very fast, so we have done case study on Matrix Multiply show doing how you can basically do things like algorithmic tricks, architecture tricks, and stuff like that get really good performance on Matrix Multiply. So we started with doing very object oriented, immutable type matrices, Matrix Multiply and then go down to doing blast. How much a performance gap do you think that has?

AUDIENCE: It probably depends on how big the matrices are because--

SAMAN AMARASINGHE: Yeah it is 2k by a 2k matrix so it is, yeah-- So somebody said 200x.

AUDIENCE: 10x

SAMAN AMARASINGHE: Yeah 10x, so actually, in fact we've been-- since I started with the immutable matrix, very object oriented stuff like that, and then did all to optimize I could, I got 300,000x. So this is kind of extreme case, but it illustrates how much, kind of, performance sometimes is hidden in these applications.

And then we talk bunch about, these are done lectures, talk about some very simple things in performance engineering, how to optimize loops, things like how to get the compiler to do certain things. Some basic facts, we have bunch of rules in there that we went through fast. And we are going to do introduce them to performance analysis for a lot of these people. First of all, they don't even know their code is running bad, because they don't know how to look at it other than getting one number then.

Even if they know that their code is running bad, how do you go about figuring out what's going on? They know a little bit about GDB but they don't know anything about performance analysis, so we are introducing them to tools in here. And we are talking about C to Machine Code, lot of them are Java programmers, we already gave them a C primer, saying if you're Java what does C mean, kind of a thing.

And I actually talk about computer architecture today and talk about the entire process they're going to use. They're using this Nehalem processor and all of the interesting things that happen inside the processor that they had to be aware of. And the same, maybe I will talk about the memory system optimization, and then Charles is going to talk about algorithms and data structures for caches in there and some about storage allocation.

We'll have a bunch of lectures on parallelism that Charles is going to use Cilk Language to illustrate how to write parallel programs, that's what we are going to use in this class. And then more on parallelism, some things like data synchronization issues, dig deep into other performance issues in parallelism in there. And then since their final project is on Ray Tracing, we'll introduce them to a Ray Tracing primer, and see how Ray Tracing is done and so they can go into their final project. Then I'm going to talk a little bit about compile optimization, what compilers can do for you, and how to get compilers to do those things for you.

And a little bit about distributed systems, so how can you scale out on this point? It's about getting a small piece of code to run fast, now how do we get millions of users go to this very large scale system, and get them to work, and what issues are involved. And then we are going to also have about three or four case studies sprinkled around.

We'll get someone from University come and talk specific problems, OK, look I had this problem, and at the beginning it was bad. Here's how I went about figuring out why and here are the things I tried, these are things worked, these didn't work, and how we went go. So kind of a very hands on case studies about doing performance because what these people has to do is develop a methodology. Their own methodology of how to go about solving these problems, and then looking at these case studies will give them a feel how a very experienced person will go about looking at some of those problems. So these are the kind of lectures we are doing in a nutshell.

So the projects we are doing. So students' mission, what we do is we write some inefficient piece of code, not that long, and we give them the specific, here so this is functionally correct but it runs darn slow. And what you can say is, OK, take advantage of right now, the first project is basically looking at algorithmic and data structure inefficiencies. So OK, trying to find better ways of doing that and change this program to run faster. So that's their thing, so they don't have deal with thousands of lines of code. Small program, it's correct, to run them faster, and this we'll also be asking them to write tests, so you can actually test their implementation, and make sure that implementation are also correct.

And basically there's no right answer. This drives students crazy because they are like, "When am I done?" I'm like, "You're never done until the deadline," because you can keep doing more and more and you could keep increasing performance in there, and of course if you hit very marginal returns and at some point and you might decide, OK, that's good enough.

And also more times the journey is as important as the outcome. So at the end of the day you can say I got it very fast, but how did you get there? What's the process you followed? Some of these programs are easy to get there, but if you haven't learned that process you can't scale it out.

And so, a lot of times your part is to talk to them and say, "What did you do?" And say, "Okay look why did you skip that stage, why didn't you profile it?" And get them to get do that. And some will say, "Yeah, I just look at it, I realize what it is." Like, no just do something there might be something hidden in there, so get through the process, understand the process, that's the important.

So the way the project works is, we give them a project, we start them the project, and there might be, some projects have two parts in there, and final and a project request design document for them to give us, saying what they figure out what's bad and what kind of optimizing they are thinking of doing in this project. And then we have a thing called a beta turn-in. They said OK look we ran this, we got the project and turned the product in, an we call it a beta, because I'll tell you why. And then what we do is, we run everybody's project against everybody, basically we run every project, and we figure out who got the fastest implementation. And we said OK, we announced this and said this is the fastest implementation, and then we figure out how off are you from the fastest, and we have a proportion of the grade that's basically computer graded. So we figure the fastest, you'll get 100% for something that on the other hand if you are 10x slower you get 10%. Except, the first time we tried, the fastest was 1,000x better than the slowest, so somebody got like 0.1%, that didn't work, so we are doing square root of the former so they'll be little bit more, in the somebody, got 1,000x there.

So sometimes it's because somebody just missed one critical insight and so and also they might not have done the test properly and stuff like this, so we give them opportunity. And then after this beta turn-in you get this code, and the TAs get this code to run and do a very fast check over that and that, and then they are going to have a week to find a time with you, sit down for hour or 90 minutes, and go through this code. And then they will get all the feedback, the performance feedback, and they can talk to people and figure out what they missed and they get another chance to fix all those things, get the performance up to turn in, and there'll be another performance measurement, but now if they match up to that point, they can get a full amount so of course, if people exceed they don't get extra, because we want to limit the competition, but at least you get a chance to kind of match up the best code.

So, what you have is this design review week, that they get to talk to you, and get feedback and, then after that, they fix everything up and then resubmit it.

PROFESSOR: You should mention the beta tests.

SAMAN AMARASINGHE: Yes, so what also we do in this is we're asking them also to create a test suite, and we are going to run everybody's programs against everybody's test suite, and also give them points on coverage of their suite, and also how well their program does against everybody else. So if somebody finds lot of bugs in other programs, or some there's a test that founds couple of bugs that nobody else found we will give a lot of points for that person, because that means they went and did a really good job in creating a test suite. So we are actually using that to basically emphasize on testing and testability issues in there.

And then final, when you're turn in, then the TAs will, at that point, will grade for performance and also the coding style. So that time they will go to the code and assign the grade. And so if the students have listened to you, and carried out what you said, they should get really good grades on that part, and then they had to also provide a write up in that. So this is kind of the flow of the projects we have.

And the project one is a simple algorithmic and data structure trick. So we did things like data rotator, bit flip counter, and this Pentominoes Puzzle that he's going to explain a little bit. So we give these things, that we have a pretty inefficient representation, where most of them has a much better bit representation, and you can do bit tricks to basically get good performance. So these are algorithmic and data structure representations that can have huge performance gains. So this is the first project and this is what next week you're going to get.

So project two is more about memory system performance, things like image rotator and number sorting. We have two parts, the first part is we give them these problems, and they're supposed to run with different data set sizes, look at performance counter results and identify what's going on with these programs. So there's one thing that we'll give them bunch of source compiled, and they have to figure out which sort is which by looking at performance counters and different ones.

So you had to identify, so basically give them idea to basically observe a program behavior, and then we ask them to go and fix some of the performance issues as a second part. So this is a lot about basically getting performance information, understanding a lot of hardware, what's going on and then going and doing some performance fixes. So there'll be two parts, to this project.

And the project three is implementing efficient storage allocator. Most of these students find after they've done the Java and Python, they come thinking there are these objects running in memory, with a thing called pointer that points to, and to understand what a pointer is is sometimes hard. And this is an interesting example that they actually do a lot of pointer manipulations, and you have to get a lot of those things right and you have to have good discipline to do that, as well as getting some performance, so this is interesting project in there.

Here you don't get too much of performance variance but students have to think very carefully about basically aliasing data structures and stuff like that. So getting some of this dirty little things you have to do, and learning how do it right. And then we go into parallel programming. So first part is introducing the concept of parallelism giving them some simple programs to basically parallelize, and then the second part is we give them insertion detection we're inserting a bunch of lines into a image and then they have to figure out when these lines move when they actually intersect, and so it's not embracing a parallel, you have to actually do by intersegmenting the data into regions and basically figuring out where communication is.

So there's a lot of issue in there to get it done. It's a fairly complex program that they have to to deal with and so they're going to have to think through a lot of parallelism issues, synchronization issues, communication issues, all those things come into play in here. And then project 5 is just a problem set because we want to get to make sure that they understood some basic material we did, so you don't have to deal with that. We give them, actually, a write up they don't have to do any coding in there and they turn that in as a normal problem set. And the project six also you don't have time to critique that because it is delivered the last day of class, but if you're interested in basically giving them some feedback, or, just come in to the finals and see how they do.

So there what they do is, this is a really fun project, we give them a very unoptimized ray tracer, and we okay here's an image that it can generate. If you generate image that looks the same to us, you'll be OK. So you can change the algorithms you are doing, come up with other things you can do, where you can still maintain the visual equivalence and then you can do that, you can optimize, you can parallelize, you can do anything you want and build a ray tracer. So they go and then we explain to them the basics of ray tracing and they do lot of cool things and a lot of these kids actually really change, think through algorithmic issues, even look at a lot of graphics and find different ways, or better ways of doing things, and come up with really cool tricks and then do really well. So for that we will invite you guys to come to the-- then we will have basically a bake off, everyone will come and figure out who has the fastest ray tracer, and they have the bragging rights, as well as the grade of having the fastest ray tracer, so--

PROFESSOR: I'm going to go through this.

SAMAN AMARASINGHE: Okay, you go through-- we have the schedule here. So the most programs are done in C, and very little C++, not very complex C++ classes, or anything like that. We want them to be very close to the metal. They have been trained in a lot of abstract programming in Java, Python and stuff like that, now they're going to get to the metal, and understand things like pointers, malloc and free, native data types and stuff like that. We're not dealing with things like dealing in garbage collection and bound checks, and all the other things, so they're really seeing what's going close to the metal, and sometimes we are even encouraging to look at at a little bit of assembly work, that's not really required.

AUDIENCE: Have these students had exposure to any kind of assembly language programing?

SAMAN AMARASINGHE: In 004, they are using some assembly programming for in the architecture class, but I think it's a very simple, I think RISC machine inspection they're doing. And then we will have our course website as you go about you can look at what the course is doing and also if you change this F10 to a F09 you will see last year's code. So you can actually even see into the future what we have been doing. So if you want to kind of preface from the front.

AUDIENCE: I took a look at this website and found that most of it was unavailable because of--

SAMAN AMARASINGHE: We should have Word readable we need to make it if it's not Word readable--

AUDIENCE: It says it's Word readable. To us.

GUEST SPEAKER: Some of it is a lot of it was not I've said you need to, I mean it's all available for-

AUDIENCE: Certificate?

SAMAN AMARASINGHE: We will check that, because all this things that we have set up in the background to get everything Word-- we want to make everything Word readable, so we'll make sure that happens. So, OK, so that's all I have about the class and one thing interesting more so this projects, the best to worst can sometimes be 1,000x difference. So what that means, even for the people who are struggling, they can get something out of the class.

But unlike other classes where you just read the limit and then you're bored, you can really keep pushing the limit, especially things like the final product, the people who did well really pushed the limits, they could have done that for a graphics class type thing. I mean they pushed algorithms changes, did amazing parallelization techniques, data partitioning, everything. And where some people just kind of stayed within that and did some small parallelization got some like low hanging fruit in there.

I mean what we're trying to do is, computer science these days, we get some students, as I said, their first time they're coding anything more than ten lines and other people who are already a submitter for open source projects, who have written millions of lines of code, we have these two. And these codes, I think we are trying to cater to at every level, and keep everybody basically very motivated, and excited to be in there. And I think the last year it was a pretty good success. And they really enjoy meeting masters last year, and after we go to discussion we can talk about that a little bit.

PROFESSOR: So we have two websites, as Saman mentioned before, so here they are, and then here's the schedule. So we'll run through this, in more detail for what is expected of you, in a moment, but basically there's four projects that's sort of are organized along here. Oops, that typo didn't get fixed. That's should be November 19 obviously there, instead of October 19 if you want to correct that on your calendars.

OK, so in general it begins with an information session which you're allowed to attend in person but most people just dial in, it's more convenient. Then we will provide you with the beta code submitted by your four students and then you'll have two meetings that typically are 60 to 90 minutes with each of your two pairs. OK, so let's go through how this is working. So the first thing is just to make sure you folks recognize that you're making a commitment. It's very difficult for us, and it's even more difficult for the students, to cope with something that happens mid-term, OK?

Where suddenly their master disappeared, or whatever, so we're very big on anti-flakiness, if you will. And so if you believe that there's going to be a problem please let us know as much in advance as you can. Sometimes things are unavoidable, they come up, we don't fault you for that or anything, OK? Just let us know, we'll try to deal with the situation. But for the most part we're really asking you to make a strong commitment to the class, and so if your situation is such that you're unsure it's best to talk with us or politely decline to participate, rather than putting on the rosy glasses, say, "Oh yeah I think I can do this, even though I know that I've got a major deadline that conflicts with one of these projects," or whatever.

It's just really difficult on the students when things change. They are sort of going I don't know if you remember what it's like being in school as an undergraduate, but pretty much the students go sort of hand to mouth, in terms of like, they look and they say, OK I got this deadline this week I better work on that one. They don't really look ahead and so forth, and so it's up to us, to some extent, to recognize that they're in that mode and that we need to do things to make that easy for them. When they get sick or something mid-term it can really throw a student way off for the term, and so certainly when something happens to the staff, that sort of, at that caliber, can really knock them off their trajectory.

The design and code reviews should normally be held at MIT, and the students will have some locations. If you are within walking distance to campus however, you may suggest an alternative place to meet, such as at your own workplace, if that's more convenient for you, OK? But if a student balks at doing that you shouldn't pressure them to accept. The content of the off campus meetings should be professional, it's strictly the review process, no lab tours, no free lunches, or dinners, or what have you, OK? And we'll talk a little bit more about that later.

But the main thing about them is that they can be scheduled in the evening or you know whatever time you and your two students find convenient to get together, OK? There's no limits as to when you know what time those can be. They can be on the weekend they can be, whatever you can work out, very flexible.

So here's the basic guidelines, each master is responsible for reviewing the code produced by four students. The students typically will work in teams of two or three for the projects, OK? So what you will have is your four students will be organized into two review groups of two students each, but none of the students will be on the same project.

The reason for that is that we want to make sure that every student who goes to represent their project can represent the whole project. So they're responsible for the whole project. They don't have a buddy there on the same team, who can cover for them when you're asking them questions about their project. You ask them questions they've got to be able to answer about the whole thing, even if they didn't code every piece of that. We're encouraging them generally to do things like pair programming, so they really should understand what's going on throughout.

The reason for having two groups, rather than meeting them individually, it turns out there's actually some good knowledge of one of them sort of being able to see what the other teams are doing. So generally they're not allowed to share their solutions, et cetera, with other teams, but in this meeting that's OK. The ideas and so forth can come but they shouldn't be making copies for each other of the code of the other group. But they can say, oh is that what the other team did, And so forth, and there can be some learning for them to submit the final project.

SAMAN AMARASINGHE: Also last year, lot of masters very successfully used the other team members to kind critique each other. So showing something do you understand this, and then they can look at it and say, "Yeah, I can see what's going on, that's very useful," also when we go to masters we can talk about of that type of thing.

PROFESSOR: Also things like documentation. You can look at the code there and ask the other one, "Do you understand what that documentation says?" And so forth. It turns out it's actually easier than that, because usually if the documentation is inadequate, the student himself has trouble remembering what they did, right? I mean you know how this works with code, so.

So consequently you're responsible for generally for two review meetings for each of the four projects. Now as it turns out we'll assign things, we're going to have things like students dropping the class, and so there will be some minor reshuffling as we go on. Some of you may end up with only one team of two students.

Some of you may find that you're reconstituting it. In some cases if we end up-- we may ask somebody, "Can you cover an extra team this time because somebody is going to be out of town?" Or something like that. So there may be some variations on this, but this is basically how things go.

Now, one thing you should understand is that you are not going to grade the students. So, one of the things I learned long ago was that when you grade somebody they behave differently than if you don't. So you're not grading them that has a plus and a minus.

The plus is that then this is really pure feedback for them. This is to help them, they know that you're supposed to have read it. The downside is some students will say, "Eh, then I don't need to listen, because nobody's holding a stick over my head." But the reason we do that is twofold. One is because we think that the quality of interaction is better, and the second reason is because you're not empowered by MIT to give grades. So you won't be grading them.

You're also not responsible for the technical content of the class. We're going to be teaching stuff, hopefully which is sufficiently new, some of it, that some of you may not know all of what we're teaching with respect to parallelism, and so forth. Some of you may that's great, but that's not the role. We will take responsibility for teaching the content of the class. What we're really after here is for you to listen as an experienced program developer, and provide feedback to the students about software engineering.

In other words explained to them what's going on. "Why didn't you write that in your documentation? How come all your variables have only one letter? Why is this stuff wrapping around three lines before you-- why no white space?" This kind of stuff.

So, remember they're very smart students, but they're also very immature many of them. Some of them you'll discover are like way mature, but there's going to be some of them that are just absolutely this is new to them, and what they need is some encouragement and some suggestions as to what they can do to improve the quality of their code. And since we're not in a position to review it, why we're deputizing you as members of the MIT POSSE, is to deputize you to help them produce better quality code, which is not something that's easy for us either to give feedback on, or to grade.

So hopefully when we are grading them in the end on their final submission and the quality of things, it reflects that they've learned something from you folks about producing good code. Also you want to be careful, this is one of the bugs I have when I interact with the students is, I tend to do most of the talking if I don't check myself. And one of the really good things is to have a guideline that they I don't know how many of you have teenage kids, and if you lecture them how quickly they learn right? No, they don't.

So the goal here is to help them learn. And so lecturing usually doesn't work. What works generally is getting them to do the talking.

And then it's great to pepper your interactions with anecdotes. I work with one person who did this, or early in my career I did this and here's how it blew up in my face, that kind of thing. Excellent to have anecdotes that you can give with real world experience.

Why you might think it's like this, but in fact, if you do it that way you're headed down a very bad path. Let me tell you story about that. That's fine, OK.

SAMAN AMARASINGHE: Or if you find a very stubborn person saying, "If you do that I won't hire you."

PROFESSOR: That counts too. You would never get a job here if you coded like that. And in fact that's actually what Saman says, that's actually one of the big benefits of having folks from industry providing that feedback, because we can say that, if you do it like that you wouldn't be hired by XYZ corporation. You can say that, but if you say, I wouldn't hire you, if you did that, that has a, it's like, oh! It has a big impact, whereas when we do it it's like eh, you know. It's like what do they know?

So the guideline is make them do a lot of the talking, so that they're trying to explain and you teach them a little bit what a design and code review is. Now, in fact what your review is, is both design and code review.

SAMAN AMARASINGHE: To add to that, what we have told them is to prepare for what a filing statement type thing, to walk through and explain what they did to you guys. So they will come with some preparations stuff that they can do that, let them, both of them kind of do that, and then start them off the instructive part.

PROFESSOR: And maybe they'll do that. But that's the goal is, you should say, "Look, when you come you should prepare something that brings me up to speed," and teach them a little bit about what's expected of them when you have a design or code review in industry. So for each of the projects, as we mentioned, there will be a short information session, typically by dialing conferencing, except for this one, which we'll do in a minute.

After the students have submitted their beta releases we will get the code to you. Last year we had the students responsible for getting the code to the MIT POSSE, and that was basically a nightmare for everybody involved. So this time we will make sure that we get you their code, and I hope that way the unreliability-- I don't know if you remember but like, these are mostly like juniors and seniors, and when you're juniors and seniors you're not 100% reliable in everything you do and your word is your bond is not necessarily, oh I forgot that my word is my bond, kind of thing, right?

So by us getting it to you, we'll remove one layer hopefully of flakiness. Next what you'll do is, you'll coordinate to meet with a mutually agreeable time, and then after the review you're going to provide us with a little bit of feed back, it doesn't have to be long, just a short email, not as a grade or evaluation, but to help us understand where the student's are doing well, and what they're missing. So what we may need to stress in class and so forth.

And then, of course, after the review, they have the opportunity to make changes to their code before they submit their final revision. Now, that's a little bit different from industry where you would go through typically some kind of feature freeze, and code freeze, and so forth. Our beta is beta sort of in name, but not necessarily in all aspects, in that they can actually completely rewrite their code between the beta and the final version.

SAMAN AMARASINGHE: So in your feedback you can also tell a little bit about in your interactions what worked and what didn't. This is the first time, as far as we know anybody's doing this kind of a thing anywhere. We are also developing a process so good practice is very--

PROFESSOR: And since your email will go to MIT POSSE and everybody will read it, what hopefully you can do is, some of the practices that you find useful, I tried this at work, I tried this, it didn't work. That starts to get spread out among you and is becomes very helpful for each other to say, "Oh, well that was an interesting idea let me try that." And so we hopefully we'll be able then to start compiling some best practices.

AUDIENCE: Will we be identifying students by name in those broadcasting minutes?

PROFESSOR: Yes, yes, in fact here's the thing is, what you're going to do is give, well it one, names the students who attended, and that helps us keep track of who's actually going to these and who isn't. The strengths and weaknesses in the student presentations. Some suggestions for improving the assignment. That's suggestions for us to improve the assignment for them, and any other comments.

So we got some really good feedback last year that, "Hey the code you're handing out as the initial code that was really sloppily documented." Last year we were much more behind the eight-ball in terms of trying to do this. This time we've had more time, and in particular we also have some really outstanding TAs who've worked very hard on the software engineering of the project. So I think that we'll be better in that regard, this year, than we were last year.

AUDIENCE: I just had a question

PROFESSOR: Yeah, sure.

AUDIENCE: You mentioned that they're going to give to you and you're going to pass the code out.

PROFESSOR: Yes.

AUDIENCE: Do you mean just the code or are you thinking structured more as a design review, they really will be talking about, a review package. It would be the code plus I don't know whether you're doing documentation generation or any sort of an intro or front matter?

PROFESSOR: Yeah, so the question is whether in addition to the code there should there's going to be any other documentation, or package, or whatever. In fact, most of the final documentation is going to be for the final project and not for the beta, and so they should have things well documented within the code, for example about what their strategies are, and how they work, and so forth, but there won't generally be a write up. They should be providing you at your meeting with a brief outline of what's going on, and you can ask them after the first meeting for any other thing that you think would be helpful to them and to you in making the design meeting more productive.

One of the things always in these things is how much do you require out of the students and at which time. They have in this project, we're on a relentless series of deadlines for them. And so we've tried to do as much as we can while being still relatively minimalistic in asking for too much stuff too early. And so it would be great if they had a complete package of here's what's going on, the fact is that they were up really late the night before trying to get this stuff done, they've also had to submit beta tests and so we decided that the final write up of how their project worked, et cetera, we would save for the final, and it won't be something, in English written like that will only be in the code, it won't be as a separate package.

AUDIENCE: Right.

PROFESSOR: Any other questions about that? OK, so one final word, relationships with students. So one of the things that MIT has, and many other places have, I'm sure that you're aware, that if you're in a power position with somebody, and this in particular applies at MIT with students, you must ensure that all your relationships are strictly educational.

And what that means is like a, no romance, OK, duh. But it also means in this context, no recruiting, hints at job opportunities, offers of summer internships, lab tours, et cetera, free dinners, and so forth. Your job has to be completely educational.

However when the term is over you're welcome to continue your friendship in a non-power relationship. There's no longer a power relationship you're welcome to start a romance with one of the students if you will, but in particular, allowed to seduce them to come to your company as a summer intern, or a job or what have you. So you're welcome to do that but not during the term, those sorts of things are absolutely off bounds.

Now the students themselves may say, "Hey do you guys have summer internships, and so forth?" And it's up to you folks to draw the line and say something like, "Yes, we do, but if you're interested in that, we can talk about that after the term is over," for example. So, put them off in some way, and return the conversation to the educational mission for the term. Yeah?

AUDIENCE: So what if the students do a good job?

PROFESSOR: Yes, but then you run the risk of it being interpreted as a quid pro quo for employment, which is exactly the thing that we're trying to steer away from. They're smart MIT students, I think they'll figure out that if they're interested in a summer internship they should be impressing their mentor. Not all of them figure this out, by the way.

AUDIENCE: So just to clarify. Just to clarify, it's okay to say, "If you write your code this way, I won't hire you." But it's not OK to say, "I wouldn't hire you." I just want to clarify.

PROFESSOR: You'd say I wouldn't hire you, OK, my company would not hire somebody who wrote their code like this. You could say that.

AUDIENCE: But it's not okay to say my company looks for people who would write code this way?

PROFESSOR: No, I think that would be OK too. No, the issue is whether you personalize it, right? If you're interested in a summer job, I can get you a summer job, all you have to do is, dadadadadadadaaa, and now they feel like there's another master to serve. So what we want to avoid is having them feel like there's another obligation on top of the obligation they have to the class. They should not feel that in some ways they're obliged to do something for you in order, for example, to get a job.

SAMAN AMARASINGHE: Or get a good grade.

PROFESSOR: Or get a good grade, or--

SAMAN AMARASINGHE: If I say no to the summer job they're going to badmouth me and I will fail this class or get a bad grade, and then. So, that put them in a binding situation that they won't work for the class, but they'll feel obliged to something else.

PROFESSOR: You're in a power situation, and you're in a situation where you must not let them have any sense that there's any power relationship there, if you will, in other words--

AUDIENCE: We shouldn't try to extort meals from them.

PROFESSOR: For example. Exactly, exactly. Also if you feel anything awkward is developing, let Saman, or me, this is something for which the staff only email is better than probably the MIT POSSE one.

SAMAN AMARASINGHE: Or even to us directly.

PROFESSOR: Or even to us directly, although I actually think it better to go via the staff only. It really is much better to go get via staff only, because the TAs need to know what's going on with all the students in class as well. Question back here.

AUDIENCE: Two logistical questions but I don't want to interrupt you before you finish.

PROFESSOR: No, that's fine.

AUDIENCE: Okay, it's not clear to me do we meet with the same four students throughout the term?

PROFESSOR: If possible we will do that. Because otherwise what happens is, they hear it from one and then they go to another. So this way there's some amount of continuity, but that won't happen perfectly because students will drop and we'll have to reconstitute groups and things.

GUEST SPEAKER: The winners get David and the losers get me. That's OK. I just wanted to be clear. Also you e-mail the code to us before we do the sit down session?

PROFESSOR: Yes.

GUEST SPEAKER: Are we expected to review it by ourselves beforehand?

PROFESSOR: I think it helps to take a look at it, yes.

GUEST SPEAKER: As opposed to walk in cold and doing just that.

PROFESSOR: Yeah, yes, that's right, I mean I think you do a better job if you have looked at the code beforehand to see, oh my goodness this is, and take a few notes. But for this we're relying on you to use your best judgment for what's the best way of giving feedback to the students. But I think generally the reason we email it to you in advance is specifically so you have a chance to look it over.

Once again we're not so concerned about whether the content of what they're learning in it as much as-- you look at it, it will take you two seconds as soon as you see one piece of undergraduate code, it'll take you two seconds to say, I don't care what this thing does this student needs to learn X. I mean you will see. Is that the feedback, from people who took this--

AUDIENCE: You'll know it when you see it.

PROFESSOR: Any other comments from people from last year what--

AUDIENCE: Yeah, a couple comments. One very minor sort of procedural things. Personally, I found it a lot more useful to me to show up in person for the information sessions than do it by conference call, if for no other reason then a lot of people who called in had issues with voice quality. You know, just getting a good quality voice signal over the phone.

Also I found it very helpful to have each student review the other one's code. So I would just do that to start with, just hand it to each other and here, what do you think? That also gives them some experience of actually doing a review, as well as being the person who's work is being reviewed.

PROFESSOR: That's a great technique just because one of the things that you want them to do is empathize with the person who's going to read the code, and so, by putting them in the situation where they have to read somebody else's code they very quickly learn oh, this is, my goodness that's exactly what I did to the other one, so that is a great technique.

AUDIENCE: The reference implementations are we going to get those--

PROFESSOR: Yes.

AUDIENCE: Soon? Early?

PROFESSOR: Yes, those generally we will be able to give you the reference implementations at the information session. Or actually, we should probably get it to them before the information session.

AUDIENCE: I would suggest before the--

PROFESSOR: Yeah, let's get those make sure we get those to you before the information session. We'll get you a reference implementation at least a few days in advance.

AUDIENCE: OK, also something that was a recurring issue last year regarding the reference supplementations which was the... there was some issues with some of the reference implementation--

[INTERPOSING VOICES]

SAMAN AMARASINGHE: --software engineering.

AUDIENCE: In particular things like functions that should have been local to a file they were not declared static or, that kind of thing, or macros that should have been used or that were used and shouldn't have been.

PROFESSOR: Last year we were running with a much larger class than we expected, with no infrastructure to speak of, and we had two TAs who were flat out, and so we definitely cut corners. This year the department has been good to us, we have four TAs, and so I think that the quality is going to be higher. That doesn't mean that the TAs are seasoned 20 years of software development experience or more, either. So they have some things to learn too, but already based on the first project we're ahead of the game.

AUDIENCE: OK, also if from time to time one of us wants to attend the class?

PROFESSOR: Feel free to attend the class.

AUDIENCE: OK, where's the schedule?

PROFESSOR: It is Tuesdays and Thursdays in 26-100 at--

AUDIENCE: Is it on the website?

PROFESSOR: It's on the website at 2:30 to 4:00. So Tuesdays and Thursdays 2:30 to 4:00, you're welcome to come and learn all about performance engineering. Reed, did you have a question?

AUDIENCE: No, I guess I was just going to say that in regard to the feedback, we TAs here have got that.

SAMAN AMARASINGHE: During class last week most of the TAs were students last year, so they aren't learning anything new.

AUDIENCE: I'm sorry, where is the class?

PROFESSOR: It's in 26-100

AUDIENCE: So it's not here?

PROFESSOR: No. So this is the situation, we outgrew our classroom which seated 60, and we had 85 or 90 students, so they put us into the next smaller classroom that we would fit into. That was 26-100 which fits 500. Then we found that in 26-100 you needed a microphone for the students to hear you, even though we had them all crowded into one quarter of the auditorium, you still then-- just the structure of it-- you needed a microphone, so they said, "That will be $85 a lecture." And you say, "What do you mean $85 a lecture?"

SAMAN AMARASINGHE: Someone has to walk in just to turn on--

PROFESSOR: To turn on thing thing. So, anyway we sorted all this out OK, but it was sort of funny that you're charging us for being in a room that we have no choice as to be in? Anyway it was very funny.

AUDIENCE: Are your lectures video taped are they available--

PROFESSOR: They are videotaped. We are not planning to put them up quick because what that does is discourages students from attending the lectures, as you can imagine.

SAMAN AMARASINGHE: Mainly for OCW OpenCourseWare So they basically they could go to OpenCourseWare they will process it and they have to go through a lot of legal stuff and make sure the named are right.

PROFESSOR: Are right and so forth, yeah. There is a bunch of stuff there. Good, so it says final words here but they aren't really final, because we still have Eric to talk about the first project. But the feedback from the students last year was that the MIT POSSE was one thing that made this one of their favorite classes. It was getting that kind of seasoned feedback in a very small group setting when you look at the student faculty ratio, that's one thing you're always battling. especially at MIT, where we are in a very popular major, and so the ability to recruit all you folks to help that ratio in favor of the students is really very much appreciated by students and by us, so enjoy the term. This is really rewarding the people who have done it before we had just tremendously positive feedback from them about just how rewarding it was to work with these young programmers.

SAMAN AMARASINGHE: They are very smart people, you can see the changes, you will see the you impact on them. You will see you're molding these people into better programmers, so that's the fun part.

PROFESSOR: So Eric is now going to talk a little bit about project one, and this is similar to what we will do, although most of you will probably be on the phone unless you heed Barry's advice, for the future one. So I'll get out of your way.

ERIC: So the project one this is, I think, the one project that was substantially different from last year. So, the code for this project we, the TAs, coded up this semester. Did we give out the hand outs, Saman, or Charles? Yeah so these are the--

PROFESSOR: So this is the actual assignment and we'll share this with you, plus the reference implementation, I think.

ERIC: Well we'll send out the reference implementation tonight then. In fact, now people have already done their betas for project one. Project one has two parts, it's a part called Every Bit Counts and a part called Tiling a Torus with Pentominoes.

So the first part it's an abstract data types of modeling and bit array, packed bit array and they are a given a couple of tasks to do on long strings of bits. So you might have eight million bits and there are two functions to rotate the substring of the bits by some amount to the left or right, and the other operation is to count how many flipped bits there are in a given substring. So the specific interface for this, this an excerpt without the comments from the header file of this C module. There is a couple of accessors which accesses bits individually those are not, well those accessors are currently used to implement, in the reference implementation, the rotate and count flips operations.

So what they will need to do is to go into the C module, the implementation, and change these functions, to basically be 10,000 times faster, or whatever, I can't remember what they actually got. In particular the rotate operation is incredibly slow, there's one constraint for them and we're saying that, well imagine that this is on the cell phone or something, where you have a really big bit array, but you don't really have very much more memory to consume beyond that big bit array. So we're asking them to only do this using a constant amount of memory.

So that's the first problem with these two parts, the rotate and the count flips. Any problem, questions about that particular part? Does it seem clear what?

AUDIENCE: What does the count flips do?

ERIC: Oh yeah, so I should've swap those two slides. So, if you have a bit string, suppose nine bits or something, 1, 2, 3, 4, 5, 6, 7, 8, that would have to be 8. If this is the substring you want to count, then you count transitions from zero to one, or one to zero. And there's some new tricks you can do by for instance, considering six--

AUDIENCE: Count the number of runs, of consecutive value.

ERIC: Yeah, the number of runs.

AUDIENCE: Plus plus one-- or minus one, yeah.

ERIC: That's right. And you don't count the beginning or the end as a transition, so in this particular substring there is one there, one there, and one there. And it's not a substrings, it's always substrings, it's not the entire string, and you can see that for instance, you give us an argument, the bit array and you say where the starting offset and the number of bits that the substring that the substring is and that's where the operation is done. So our testing routines will do lots of rotations in different parts of that substring.

AUDIENCE: For the memory constraint, do you have a fixed size that they're allowed to use or?

ERIC: We decided against the fixed size. We just say don't use malloc, and you can allocate constant amounts of memory, and the thing is if they choose to have a 16 megabyte or four gigabyte lookup table then their performance will suffer anyway. So yep?

AUDIENCE: What is the computer architecture?

ERIC: What's that?

AUDIENCE: What is the computer architecture?

SAMAN AMARASINGHE: We are running on-- we got this donation from both Intel and Dell, we have 16 machine-segments, each with two six core processors and 48 gigs of memory. So they're doing it on 12 cores so it becomes interesting, and you got a parallel 740 gigs, so we have a lot of memory in these kinds of things and we'll be asking them to run through that. So these were the latest what's the word?

PROFESSOR: Core i7.

SAMAN AMARASINGHE: Core i7s.

PROFESSOR: Nehalem.

ERIC: For this first project, we're asking them specifically not to parallelize.

AUDIENCE: It's all on Windows or Linux?

ERIC: Linux.

AUDIENCE: Linux?

ERIC: Yeah. So the idea here is to exploit bit operation and in particular in rotates. The reference implementation does the really stupid thing of you want sort of 4,000 bits to the right rotates, well I'll do it one at a time, that's the reference implementation. But they're only allowed to us a constant amount of memory, so there is certain amount of tricks you have to do to do it in a more efficient way.

AUDIENCE: Do you stress in the course about portable data code from one operating system to another, one machine to the other.

PROFESSOR: This is where we want you to stress the importance of it, because that's not something that we're in a position to do or enforce, or what have you, OK? We let them assume for this problem, for example, that the machine they're working on is Little Endian. We talked about the issue, but we did not insist that they wrote code that would be that would port to a Big Endian machine, because they're running on the Intel machines.

But this is exactly the kind of thing where code quality would say, portability, maintainability, readability, all these kinds of things, this is exactly the kind of thing we want you to say. You say "Hey, you did it like this, in fact did you know that without sacrificing anything in performance, you could have made it portable from one operating system to another?" And then you could talk about those issues, and that's absolutely super, to do that. But we are not spending a lot of class time doing that, we're really focusing on the performance engineering part of the course.

ERIC: The reference implementation should be portable, so this should be portable, although we did make sure here, for instance, in the reference representation of bit array here, that the order of the bits within each bite happened to coincide with the low end ideas of these machines. So that if you happen to cast it to a 64-bit integer on each of the bytes in the array then you would get something that was easy to work with on a low end in architecture. But the reference implementation is portable itself, or should be, so tell us any kind of problems with the reference implementation like that.

The next problem is based on the puzzle, it's a puzzle solver. It's this puzzle that I'd never heard of before. Apparently, if you generalize dominoes you get Pentominoes and the five end cases. So there are this many configurations of dominoes with five squares on them, I guess. And the pentominoes puzzle is you're given an eight by eight board with four squares crossed out, and now you're supposed to arrange these so that the entire board is filled up, and only-- what's that?

SAMAN AMARASINGHE: You can do flips of these--

ERIC: Oh, right. You can do flips, you rotate

SAMAN AMARASINGHE: And they will dissipate and all those things.

ERIC: Right so the task is find all the solutions, although I believe in our reference implementation that was too fast, so we decided to generalize the puzzle to the toroidal case, where the board wraps around the edges, just to make it take a little longer, to do.

PROFESSOR: Like space boards right? You go off the right edge you come in the left edge.

AUDIENCE: Top and bottom also?

PROFESSOR: Top and bottom also, yeah.

ERIC: So these two are both examples of solutions to this puzzle.

AUDIENCE: I've got a question here.

ERIC: Yep.

AUDIENCE: Are you allowed to put the empty space where ever you want?

ERIC: Well, that's the input to the solver.

SAMAN AMARASINGHE: Given these empty spaces--

AUDIENCE: OK, and then it fills in the rest.

ERIC: That's right. Of course you can put empty spaces so that it can't solved. And then they have to correctly determine that there's no answer.

PROFESSOR: That there's no answer.

AUDIENCE: Right.

ERIC: And, also I believe the reference solution takes days, for certain. To find all the solutions of certain configurations of the initial four dots. So there's an option to the program we give them, to just find the first 2,000 or something.

PROFESSOR: So once again you can see this is conducive to bit tricks, where you represent the board as a 64-bit word and then do masking operations, and so forth, to determine whether a piece fits and do clever shifting. It's easy to shift a piece down one row, where you're trying it out, because that's just a shift by eight, but how do you shift it horizontally, to come around on the other-- So they have to do some shifting, some masking-- So anyway, the first project is all about thinking about bit representations of things, and so forth.

ERIC: In fact, we give them a solver that we don't really-- well, a searcher, routine to search the space of possible solutions, that we don't really expect them to change, but they can change it if they want to do. So, the interface here is basically a board data structure that they're free to change the internal representations of.

Or did we somehow prohibit that due to testing? No? No, no, no, that's the whole point, right. So again, you have this basic accessor for use by the testing routines. But then what you really need to, well, you need to change basically the entire implementation of all of this, if you're changing the representation of the board, for instance.

But this is the function that should be running faster after you're done, so I guess that comment is misleading. This is the function that is supposed to run a lot faster when you're done. There is actual comments in the header file. So right, you give it a function pointer to a call back.

PROFESSOR: Now, one sort of standard thing we decided we're doing is, we're not trying to write both 32 and 64-bit code in the classroom, we're just going to do 64-bit. So just one more level. In practice you would want to have stuff that runs both on 32 and 64, but for the most part it's fine if they just have it running on 64-bit.

ERIC: Right, so these by the way are the initial four points that are the input to the puzzle. I believe that's the problem set one. Questions about, yeah?

AUDIENCE: Can students submit more than one answer?

PROFESSOR: No. Nope, they got to decide what they're doing.

AUDIENCE: I guess I have a question about the answer.

ERIC: Yeah.

AUDIENCE: Is an answer that is probabilistically correct, I mean like, it will sometimes when you test it, you have it pass all the test cases, but mine isn't guaranteed to necessarily always solve all test cases is that OK?

ERIC: If they can conceal that to us, then I think they--

PROFESSOR: They deserve credit.

ERIC: Keep in mind that the other half of the problem set is to write test cases that will catch as many obscure bugs in the other students' code as possible. So they run the risk of competing against--

PROFESSOR: They lose extra points if they don't pass their own tests.

AUDIENCE: They each need to write some tests that will be as--

PROFESSOR: Yeah, so then they--

AUDIENCE: They exercise somebody else's.

PROFESSOR: --to somebody else's, yeah.

SAMAN AMARASINGHE: We are running all the tests against everybody else.

PROFESSOR: And then for the final version we give them the full regression suite of the whole class, so they have very good confidence, and we good confidence that they're handing in something that is correct.

SAMAN AMARASINGHE: OK, so you want to open up for--

PROFESSOR: For questions or comments.

SAMAN AMARASINGHE: Especially, I won't even mind the people who did it last time, last time to provide some of your feedback on how we can...

[BUZZING NOISE]

PROFESSOR: Don't want to do that?

SAMAN AMARASINGHE: Yeah,

PROFESSOR: I was trying to turn on the lights. Do we have more lights we can turn on?

SAMAN AMARASINGHE: Also, if you could turn these on, too?

PROFESSOR: Good. Yeah, so comments?

AUDIENCE: I want to say that's why I was a master last year, and I had a great time, but I figured I'd stay--

[INTERPOSING VOICES]

PROFESSOR: Yeah, could you stand up so people can hear you? Thanks.

SAMAN AMARASINGHE: Give him a mic. Give him a mic because I think it'll be good--

PROFESSOR: It'll be good to have these-- yeah. All right?

AUDIENCE: All right. Thank you. So the first thing I found really helpful was to take the reference solution and actually diff what the student provide because they often just provided a solution, and if you don't know what they started from, it was hard to tell what they'd actually done. So diffing the two, that was a good place to start.

And then I found incredibly helpful just to spend 15 or 20 minutes prior to actually meeting with them, looking at the code and writing like two or three or four things to talk to them about. And it might sound like it's going to be highfalutin and stuff, but it was like they're using their returning function pointers to local variables, stuff like that. It was really like what the hell the makefiles were doing, for example, is something that I think they really appreciated me telling them that didn't get talked about in class. Especially once I got a good rapport with them, they just ask questions, and I spent the whole time answering more practical stuff about the problem. We had a good discussion, so that's what I would suggest.

PROFESSOR: Thank you, that's great feedback. Yeah, as I say, don't underestimate-- I mean, most people learn the kinds of things you're saying, but somebody told them that, hey, these are practices that really don't make sense in a real software system.

AUDIENCE: People were, by the time we got to the projects, your standard pattern was highly representative, a bunch of flags you want to encode in a bunch of fields, right? And you've seen them a couple times, it's just a standard pattern.

PROFESSOR: Yeah, so one of the things you'll find in this first one is that they're very confused about the difference between unsigned and signed, and how shifts work, and what happens when, you know, how come I shifted this thing left 33 bits and I got a zero. Why? I thought I was shifting a 64-bit value, but in fact, they hadn't done 1LL, they just did 1. And so there are things like that, where they don't-- very low hanging fruit, if you will, to talk about.

AUDIENCE: Did you tell them about valgrind by the way? Like, why would programs crash when they said they spent a huge amount of time debugging these crazy C programs when they're not used to the compiler not protecting them.

PROFESSOR: Actually, that's a good point because we were going to try to introduce-- maybe we should do that as part of the next lecture, spend a little bit of time on valgrind.

SAMAN AMARASINGHE: Yeah.

PROFESSOR: Because the memory checker there is really a pretty useful tool for them to get correct code.

AUDIENCE: I second that. That thing has saved my butt so many times.

PROFESSOR: So many times, exactly.

AUDIENCE: I've heard that, but more low-hanging fruit for valgrind is to always compile a -w all to just cache all the warning and declare all warnings as errors, and it just makes--

PROFESSOR: I think that we have that as a default in our makefile. Is that right?

ERIC: At least for the first project.

PROFESSOR: The first project?

AUDIENCE: I think they should understand that in industry practice, if you don't do that, you're a schmuck.

PROFESSOR: Right. Yep. Good, so that's good, and why, in particular, the valgrind-- we discussed that last year, and really actually, the upcoming lecture's exactly when we should do that.

SAMAN AMARASINGHE: Because there's also another tool called Dr. Memory.

PROFESSOR: OK, which makes sense, yeah.

AUDIENCE: One issue that I experienced a little last year was that people they specifically intend to write a little test for their code. They've mostly relied on a few runs by hand, then they wrote a progression producing things that they ran earlier. I guess that now you're actually encouraging them to actually have a test suite.

PROFESSOR: Yeah, well, now they get points off if they didn't write enough test that somebody else's code passed it that had a bug.

AUDIENCE: And in this case the testing run will be as far as the first turn in, and then the second turn in is just you have a version to run against and then just need to optimize.

PROFESSOR: Right, that's right.

SAMAN AMARASINGHE: So first, about the actual tests they have write a lot of tests because that will help, and the second, then you give them all the tests. So they will have a lot of test suite to run and then of course we have a lot of number of points we'll deduct of you actually fail those test. Because they have no excuse.

PROFESSOR: There's no excuse at that point, it's just a question of having--

AUDIENCE: What performance and measurement tools are you going to be using this year?

SAMAN AMARASINGHE: So, right now, we are going to give them perf because what we found was, the process of ECUT is so new a lot of performance tools don't have-- like OperaFile didn't work, because they haven't been ported properly. There's no port for it yet. So perf we are going to use, and we are thinking of giving them gprof in there. And then last year gave them couple Pintools we didn't ask them to write anything but for example, account number instruction or couple of tools they do that. Those are main tools you get, and also we need to go to Cilk, you'll do a bunch of Cilk tools in there.

AUDIENCE: Everyone last year, they were given VTune, I think it was, it was too hard for them to use.

PROFESSOR: Yeah, that was too hard for them to use because they have to use the machines remotely, and VTune and is an interactive tool, and so that just made it very complicated to do things. So instead we're going to go with a something that has textual output, and is easy, therefore, to run from command line and has all the advantages of running things that you get when you run things from command line.

AUDIENCE: So a logistical question so, meeting on MIT is fine, but where are there designated places to meet?

SAMAN AMARASINGHE: We will work with student to find places for them. For example, there are locations for PSS students to meet, and we are going to get them arranged. So if in the organizing you guys can create probably an online scheduling something. What's that website? Online is, online scheduling is--

PROFESSOR: Doodle.

SAMAN AMARASINGHE: Doodle, Doodle or something like that, and get their times and then they'll work with us to figure out the place to meet, and then they can send it out. One thing we will have to do is, between four students, because every project we are chaining the pairings, so sometimes we might end up in a way that, two of your students are paired together, and then we will tell you don't put them together in the same group, you'll basically have to mix them in the other way. If they are not in any kind of a pairing, then you can organize the four in any way you want.

PROFESSOR: Actually I don't even want of the four of them, I don't even want us pairing. That shouldn't be. But we'll see what happens. Barry?

AUDIENCE: Last year of course was in classroom downstairs and I just met them just outside the classroom. There's plenty of little space.

PROFESSOR: Also the first floor of STATA, I mean we've had a lot of people just using the first floor of this building. There's lots of cafe and place there, and so forth, to sit there, get a coffee, and so forth. So some people just did something that informally, other people wanted a more quiet private place to talk. And I think that we'll help the students facilitate that.

SAMAN AMARASINGHE: And the key thing is, especially if any students are flaking out, for example, doesn't answer your email, or missed the meeting, or something like that, or late to the meeting, let us know. Because that's part of their grade, even though you're not grading them, if they are not participating in working with you guys, they will get points deducted. So we really want to know that, and we have tried to emphasize that they have to act, and actually do it right. So we really need to know that, if the students are not actually cooperating with you guys.

PROFESSOR: So I hope that this year will have a better-- So the vast majority of people last year had no trouble, but we did have a couple of people for whom it was one or two students who just made life a little bit miserable for their master, and so hopefully we'll provide a few more incentives this year to avoid that situation. But they're students and so they--

AUDIENCE: When do we find out which students are assigned to us?

SAMAN AMARASINGHE: Probably the next two days. OK, and then who's responsible for contacting whom? That's a good question. Who should we contact?

PROFESSOR: We decided that, didn't we, at the meeting? What did we decide?

AUDIENCE: This is what happens when you don't write things down.

SAMAN AMARASINGHE: You have to cover the old stuff.

PROFESSOR: Let's see, so last time I think we had it that the students contacted the mentor, and then--

SAMAN AMARASINGHE: What we will do is we will set up, we'll send the intro email to everybody, we'll send the intro email to the mentor, Cc the student then saying I am introducing you to each other. And then one thing you can do is immediately reply and, if you have Peter Doodle Paul, or something like that. And then say, OK, look, do you know your students? So you can reply to that and hopefully they will reply back, and proceed. So when we send you the names you will also see the emails of students.

PROFESSOR: Are people familiar with doodle? Anybody not familiar with doodle? So doodle is a website where you can basically enter a whole bunch of times, and then people enter their names and mark off which times they can make things. It's a poll source.

SAMAN AMARASINGHE: Don't do what I did the last time, I marked off all the times that I cannot participate, so I was basically the exact opposite.

PROFESSOR: It's green and red, is what the colors they use. So green means you can make it, and red means you can't. So, anyway you don't have to use that facility, but it is a convenient one to use.

AUDIENCE: Can we expect students to be dropping?

SAMAN AMARASINGHE: There will be people dropping. They just started. This is also a group project and at 2:00 AM we got mails saying OK, I'm dropping the class, and we had to scramble trying to add a 3:00 AM to create another group, and so, Yeah.

AUDIENCE: You shouldn't take dropping as a reflection on our end. People drop out.

SAMAN AMARASINGHE: You have to know that.

PROFESSOR: You can if you want.

AUDIENCE: I had like three people drop

PROFESSOR: No, that tends to be more--

AUDIENCE: I had one student, it was awesome. So easy.

PROFESSOR: Yeah.

AUDIENCE: To get an idea of the mindset the students are going to be in this sort of a really tough 6-170 type thing, or is this one of the easier classes.

PROFESSOR: No this is a challenging class.

SAMAN AMARASINGHE: So you take 6-170 that has about four projects. We doubled the number of turn-ins because each project has beta and a final, and some projects has two turn-ins in there. So basically, they have all their big projects going on, because they are doing the new project and they have submitted final on the previous project. And they're already completed project zero that was--

PROFESSOR: Just like real life huh?

SAMAN AMARASINGHE: We gave them project zero. That was built to to get used to the system and we can help them with the class, they have to finish this one. So this class just screams excess very, very--

PROFESSOR: Yeah, and of course they're students, some of them have learned you put off things to the end, and haven't learned the lessons of well not always. So as I said, as a group they're relatively immature when it comes to some of these things, but there's a large fraction of them who are incredibly mature about these kinds of things. So our job is as lecturers in the class is to make it so that the students who start work early don't get penalized.

And that's probably the hardest thing in the software related class because, the classic thing is that they want to get started, they find all the bugs right? And they waste their time doing it, and if you do it later, then the TAs have worked all the bugs out. So one of our challenges is to make it so that students who start early are rewarded by being able to have more time not wasting their time.

SAMAN AMARASINGHE: And today was the day that they turn in the project and by looking at the class, you realize they haven't gone to sleep, and I asked and most people haven't slept in about 30 hours. They have been up entire last night and an email conversation when I went to sleep there was an email train going. Every five minutes, something pop up. When I woke up, the train was still going, and amazingly, there were a couple of TAs up and answering all through the night, and that kept continuing.

So these kids they can be flaky sometimes, but they're amazingly hard working. And this is only one class they're taking. They might be taking four classes like that.

And I have seen people scheduling their schedule as by hour, scheduling and they say, OK, look I can do the problems set or I have one hour of sleep. And a lot of times that's their thing, and then sometimes they triage. That's why sometimes they miss classes, they realize, OK, I can do the problem set, I can sleep, I came do that. So I think these kids work amazingly hard.

PROFESSOR: I do think it's helpful to also counsel them on, hey, you can't do everything on a death march. You got to learn at some point what you're capable of, and manage your own time resources and so forth. Because the joke at MIT is, classes, friends, sleep, pick two.

The other thing, let me just say, is that despite the obvious sleep deprivation we saw in the class today the number of students who came up after class so charged about working on this first project was really quite amazing. They were all very, very excited. And very excited we also gave them the initial feedback on the performance, and you could see, there's one group of students who clearly got it, and then there's another tier of students who didn't get it quite this much, and then there's the ones who their code has errors and stuff. There's no doubt that there are a lot of students who are very charged about this. This is a fun class but it's a hard class.

SAMAN AMARASINGHE: I think you'll find this is true. They will be, probably tired, but they will be soaking up this information like a sieve, and sometimes I found last year's-- After the looking at the first set of code, some of those the masters were really upset. How can you teach you guys, these guys write this kind of bad code, but when they realized after they said something, this is getting into their heads. I mean, they're improving they're actually changing.

These guys are very smart, and they're looking for you guys for-- They're basically sponge, really empty sponge. They're going absorb a lot in this class.

PROFESSOR: I have a 2 and 1/2 year old at home, and it's amazing what the progress is she's made in six months. She talking now in complete sentences, and so forth, it's just really amazing how how she does it, and these students are like that. They start out and you say, oh my God, they have no skills whatsoever.

By the end of the term the progress that they've made is really astounding. You just say, gee if you could just keep on that trajectory your whole life. But really, they really are very smart, and they come up to speed very, very fast, especially given good counseling. The main thing that they need right now is just the attention so somebody actually can look at their code and say, you know you did it like this it would be better code. Any other questions or issues? This is lots of fun. If you run into any troubles, we're right here, just send us email.

And as I say they're young and immature, but then you get to see them grow, of course some of them will flake out on you, but the vast majority, you'll see, are really very dedicated and well meaning. They really want to do a good job in the class, and really, really find the class very exciting. Our numbers, in fact, almost doubled for last year. So the reputation got around that this was a good class to be doing.

And the other thing is, right now what I really like about the class, the students right now compared with say 10 years ago. So 10 years ago we had the Internet boom, and there were all kinds of students taking computer science because they thought it was a ticket to fortune and so forth. And now we're back to what it was pre-2000, 1998, whatever.

Where the vast majority of students who are taking the class really enjoy engineering. They really enjoy the software. They really enjoy the engineering. The fact that you end up with a pretty good job and opportunity sometimes to do entrepreneurship, and opportunity to explore things, it's sort of like, oh that's a fringe benefit. But what they're really after is just the intellectual excitement and so forth.

So it's really quite fun to work with the students and quite rewarding. Really very nice type of class to have right now, where the students taking it because they're actually interested in the material, not because it buys them a bread ticket, meal ticket, whatever. Yeah, question?

AUDIENCE: The standard, is the coding standard in the reference, the standard is to consider for them provided?

SAMAN AMARASINGHE: I don't think we have set up a standard, per se. What we want them to do is kind of create their own style. It's like in a writing class, you don't give them a standard writing guideline is to follow this one, you don't constrain them to that. We want to give them the opportunity to grow into their style, but kind of guide them, so that they don't make the classic mistakes.

That's why we need you guys, I mean, if there's a standard, we can come up with a standard, create a script to actually check the standard, and then say, OK, thou shall follow the standard. And you can do that and everybody will hate it. You can't make that bad happen. One thing you want to do is make them lifelong good coders, not something they realize OK, I need to do this to get this grade, and I'm going to abandon everything and convince myself never to that again.

And we have had sometimes, like when we needed all this software engineering. We went through things like thou shall write all the preconditions and postconditions, and everything will be graded. And people did it, but half of them got into them and said this is good things. Other half, they didn't realize why they're doing it, they just hated it. And I don't think they left assuming I would never do this again.

So the key thing here is, we don't want force something on to them. We want them to come through a natural process, and I think a lot of them, can, when they understand what they're doing, they'll develop a really good kind of practice.

PROFESSOR: It's also the case, I think, that coding standards vary culturally from place to place, and the what people expect in one place is different from what thing. So being too prescriptive means, as Saman is saying, that it becomes a grammatical thing, rather than is this code organized. The real thing, coding standards are very important, especially in large software organizations of course, so that everybody can tell what's going on in any particular piece of code without having been the author. But in this kind of thing where we have something that's relatively small, what we're mostly interested in is the organizational aspects of coding style, as opposed to the grammatical ones.

SAMAN AMARASINGHE: So also don't impose your standard and say, my company this is what we do, I want to follow that. Just figure out and kind of guide them towards something while they're maturing, and try to make it better. That's the thing. That's why you cannot have a cookie-cutter model.

PROFESSOR: And why it's good to have senior software engineers who understand what's really important, as opposed to what might be required in any particular individual situation.

SAMAN AMARASINGHE: One other topic I want to add is sometimes when you have two students, there might be a case that one person might be very talkative and dominating, and one person might be just a listener, and so you have to keep an eye on that and make sure that you give the other person opportunities to talk, and because it might be just like that. Encouragement and stuff like that, so make sure that you get both of them to talk.

And also the are supposed to be responsible for all the code. You should not let them get away by saying, oh my partner wrote it, so I don't understand. So that's not a good answer. So get them to actually go through that, and if they don't come prepared tell them that, and you can give them some feedback, as a next time can you please do, this and this. Because they also don't know what to expect, and we give them a little bit of guideline, but it'll be good, also for you, the first time around, if you find some mistakes also for the next code review you can encourage them what to do, how to prepare and what to do.

Don't give them too much work, because we're already giving them enough work. Don't ask them to write extra documentation, but you can say it would have been nice if you had this level of preparation, if you could have explained to me, or given me a summary, or something like that. So something like that is something you can expect.

And the other thing is if you had to figure out how we want to do code review. So, for example, you might want to bring your laptop, or ask them to bring a laptop in there, if you want each other to look at the code on the computer, so ask them to bring the laptops or something. So think about the logistics of how you're going to do that, and that'll be useful.

PROFESSOR: And I think any other sort of expectations you have, in your email to them the first time say, here's what I'd like you. I think it's helpful to say I'd like you to come prepared to give me a five minute update on how you did each of the problems, for example. We will be telling them that, but to have it come directly from you means-- The difference if I say go do this to that person, versus you're coming to me and expecting it. Good.

So, I want to once again, I can't thank you folks enough for being willing to give your time. Time is the stuff life is made out of, so I really appreciate your willingness to volunteer your time for these students, and I really think you have a dramatically meaningful impact on the students. The students may or may not fully appreciate that, OK, but Saman and I really appreciate that, and we will make sure the students appreciate it.

AUDIENCE: Yeah. I just have one. What what are the guidelines for a contact outside of the code reviews with the students. I mean are they, can we have them email us with questions about the process and just kind of general guidance

PROFESSOR: It's your level of tolerance we don't want you to have to commit more than about 20 hours across the term, because then it becomes onerous. But if you're willing I think it's completely open ended, in terms of how much else. I mean it's one thing if you go into solving their problems for them, that we don't want you doing, but when it comes to, especially the quality, and non-metrical aspects of programming, the more you can give them the better, OK?

Because it's only making them be better programmers. And I think the fact that there may be some unevenness in that across the masters is irrelevant. Really, I do not want to hold back some students from getting good mentorship because somebody else is not getting the same level of attention, et cetera. But really, no expectation there, on our part, to do more than the code reviews themselves.

SAMAN AMARASINGHE: So, that said I want to make sure that you aren't answering their question, at some point I don't want to be a crutch for them. For example, if they say, OK, before prebeta, they say OK, this is my next problem, can you on a little bit, I didn't understand. If you have that, so they're trying to get too much into that level, kind of making you their programmer, or something like that, just let us know.

A lot of these things, there will be a lot of cases that arise that we haven't thought through. The students are ingenious enough they will find a lot of loopholes, a lot of kind of interesting things that we never thought through. So if you find something that you're not feeling comfortable just send a mail and say OK, take a look.

PROFESSOR: It really is, if their questions are about course content it's not appropriate for you folks to be dealing with, really. OK, that's what our TAs are for, and course staff. If it's about coding style and engineering practice, and even time management, or anything that's sort of the non-technical aspects, but things that have a strong impact on technical, that yes. Anything you can provide there is great.

SAMAN AMARASINGHE: So if you find anything that's outside the norm of what we discussed if you feel a little bit uncomfortable, just send email. Send an email, and say, hey, can I do this? Is it OK, for me to do this? And then we can always provide guidance, so we are responsible.

PROFESSOR: --do something and say after the fact, oh, it went past, let us know. There's rarely anything you can do on one shot basis that is heinous or unrecoverable from, or whatever, so.

SAMAN AMARASINGHE: It is always good, I mean that's why-- since we see everybody able to and we have done these kind of situation before. We can just easily tell you what.

PROFESSOR: But, mentor them, for sure, mentor them.

SAMAN AMARASINGHE: I think a lot of times, when you talk to students last time they got a lot more information out from the masters, other than just the coding style. I mean, they talk about what is it like to be an engineer in a company, what are the things? And so they get a good feel. I mean, some of them might not have done a internship in a company. So they don't know.

I mean, they're just green, and talking to you guys they get a feel for what it's like. And also from a non-marketing situation too. I mean you might have ulterior motives to market your company to them, and stuff, but this is not that they're going into a job interview.

So, for a lot of them it's rare. I mean they might not have retinues of friends that they can go and talk at that kind of casual level. Most of the time they're on guard because they're on an interview or they assume that information coming to them will be sugar-coated, and here is a way that they can get-- I think it's very good for them.

PROFESSOR: Eight minutes over time.

SAMAN AMARASINGHE: OK, I think we should.

PROFESSOR: So, any last final comments? Saman and I will hang around for a few minutes, for anybody wants to chat more one one. And one and once again I thank you for all the time that you're planning to put into the 6-172, thanks.

AUDIENCE: Thank you.