Although my official title is "veileder", which translates directly into something like "the person who leads the way", I am more of a "veiviser", which translates directly into "the person who shows the way". I cannot LEAD very many of you anywhere, because I simply do not have time to go all the places (i.e., learn all the things) that you will go in the next year. I've been to some of those places, and stayed for varying lengths of time, but not, in most cases, as long as you will. In short, I can point you in the right direction, but the rest is up to you, almost exclusively!
This may be the first time in your academic career that you have received so much freedom and responsibility at the same time. I was in your shoes once, and I know that it is not easy. Self-discipline is extremely important in this situation. 12 (or fewer) months is not a lot of time for the task that lie ahead, so get started NOW and keep focused on your project and the necessary background topics.
Your basic job is to investigate an interesting cutting-edge area of artificial intelligence (AI) and understand it in a) enough breadth to both write a 10-15 page chapter on the main research in that area, and to relate your own work to that body of previous research; b) enough depth to implement a system from that field, either one of your own invention or one based on the work of others; and c) enough practical sense to apply that system to an interesting problem of your choosing, but of general interest to researchers in your selected field of study.
I do advise other students during the office hours, but they have the lowest priority: I only work with them when no masters or PhD students are waiting.
You are NOT required to come to my office hours. I STRONGLY recommend that you show up a couple times a month, so as to keep me informed of your progress and to get any help that you may need. But it is your project, and you are solely responsible for the results. I help when asked, but I do not go around hunting for my students.
Students who are working with an external advisor (e.g. at Equinor or Telenor) will probably be getting most of their technical advice from them. However, it is still very wise to stop by my office hours on occasion just to make sure that you're on the proper trajectory toward IDI's two milestones: the midterm (for engineers) and the final masters thesis. The goals of an external advisor and the goals of IDI are not always the same, and it's IDI's goals by which you are graded, so don't lose sight of them. Stopping by my office hours is a good way to avoid any problems in that regard.
If you run into any problems with the external advisor(s) that you cannot address with a one-on-one discussion with them, then feel free to "complain" to me. I will then get in touch with the external advisor (if you so desire) and try to solve the problem.
All feedback on your work, including the content of your written work, will be addressed during office hours. I do not read your entire midterm report (nor thesis), give feedback, and then turn around and read the updated version again to give you a grade. The procedure is for you to come to the office hours and ask questions such as, "Is this the proper structure for the document," or, "Does this 2-paragraph description of a piece of related work sound good?", or "How do these research questions sound?", or "Is the presentation of this result ok?" If, during an office-hour visit, I find that I need a few minutes to read a section of your work (such as the motivation for your work or the description of your model), then I may ask you to leave the office and come back in 5 or 10 minutes. But in general, the entire feedback process will happen during the office hours.
If you decide to "disappear" for a semester or two, then you may sacrifice the opportunity for significant interaction with me. Although I can arrange master-advising meetings in Zoom or Whereby, they normally need to fall within my normal office hours, something that can be a problem if you're in Australia or Los Angeles. In those cases, email may be our prime interaction medium.
This year, 2022, I may relocate (to the USA) for a few weeks in late autumn and then again for 4-5 weeks in May-June of 2023. Our interactions will then be online but otherwise should continue as normal. I will be in the Eastern time zone, which is 6 hours behind Norway.
This involves reading A LOT of background material: you should expect to read several hundred pages of technical articles within your field. Conference proceedings are a great place to start: each article is only 6-10 pages long, so you can cover a good many pieces of related work in a day or two. Journal articles are also useful, but they tend to be longer, so do not get too hung up in a journal article unless you find the topic very interesting and relevant. If a conference article interests you, then there's a decent chance that the author(s) also have a longer version in some journal.
Avoid the temptation of implementing a system before you have done a good deal of background reading. You may end up building something that addresses no research issues (that people in your field recognize as important), or you may "reinvent the wheel". If you can hack up small tests in a few hours, that's fine, but don't spend weeks programming until you've thoroughly investigated the literature in your field.
During the write-up, you will invariably come across problems with your system, some of which are minor but some that require changes to the system code, reruns of the test cases, etc. This code rewriting is almost unavoidable in a computer-science project. BUDGET for it when you plan your project! Don't assume that you can only implement right up until a week or 2 before the deadline, when you plan to begin writing. The deep thinking and analysis of one's work during the write-up will almost always uncover flaws. Many of these flaws require only minor changes to the code, but the rerunning and reanalysis of the test cases can often take several days (or weeks).
For the school year 2022-2023, my general recommendations for all of my masters students are the following:
Very general background information about the PROBLEM area (not the SOLUTION techniques) should also appear in this section, particularly those aspects of the problem that motivate your research.
Throughout the thesis document, you should refer back to these research issues just to make sure that both you and the reader understand how each aspect of your work relates to them.
Research questions (RQs) are different. They are not known to be "confirmed" by the basic act of "following through" on your thesis. In fact, you might end up "refuting" them, which is perfectly fine. I prefer to view these as "scientific hypotheses" that you will be testing in your work. If this were biology, for example, then you might formulate a scientific hypothesis such as, "Doubling carbon-dioxide levels in a greenhouse will increase plant growth but also increase methane production, " as the following RQ: "Will doubling CO2 levels.... increase plant growth and methane production?"
Since our work is often less science and more engineering, the role of experimentation is often reduced. We build cool stuff more than we test hypotheses. So to get RQs out of a building process, we often need to consider alternate paths or designs and hypothesize that ours will, in some way, be "better". This could lead to the following hypothesis: "A DRL drone controller outperforms a standard PDI controller on Captain Clayton's Classic Benchmarks (CCCB)". As an RQ, it would then be: " Can a DRL drone controller beat a standard PDI controller on CCCB?" Then, it's up to you to define performance measures for the CCCB.
We can't all build "better" artefacts, especially in the course of a 1-year masters' thesis, so it's often wise to formulate RQs in more qualitative terms, such as, "Can a DRL drone controller enhance maneuverability?" Now, it will be up to you to give reasonable definitions of "maneuverability" and "enhance". Maybe you'll take a a standard PDI-controlled drone and play with it for awhile; then you'll play with your DRL-trained controller and give some qualitative assessment of the difference in "feel". More convincingly, you'll get several people to do the same thing, record their verbal responses, and maybe include some obvious data, like how often they crashed the drone using each controller.
In general, the typical RQs involve a comparison at some level and in some form (quantitative or qualitative). These comparisons should reflect an initial hypothesis that whatever you're designing is somehow a step forward for your area of research (e.g. DRL) or for the problem domain (e.g., drone control). For example, if you're working with neural networks, the step forward could be performance related (faster learning, better generalization, etc.), or it could involve the more qualitative goal of improving "explainability": maybe the nets that you design facilitate easy interpretation of the "meanings" of different deep-level neurons, their firing patterns, etc. These are both improvements to the research area of neural networks. Alternatively, you may be applying the neural nets to some domain, such as fish farming, and you decide to hypothesize (via RQs) that neural nets will make some improvements in that domain.
Again, the main thing is that your RQs are things that you do NOT know the answer to ahead of time and do NOT even know if you'll be able to confirm until AFTER you have designed your system and tested it.
Opinions vary here, but I do NOT consider this a reasonable research question: "What have other researchers done in the field of DRL-controlled drones?" This, to me, is not a question that we have any a-priori doubt about. We know that a standard literature review will answer this question. We don't need our own designed artefact and experiments to answer it. If the literature review were the ENTIRE masters' thesis, then this might be an okay RQ, but as an engineering student, you are required to do more than just read and review literature. You build stuff, and your RQs should relate to what you are building, its potential impacts within the field of research, and how you intend to apply your design to an interesting problem.
For example, if you build a simulator of some physical system, the proper level of implementation to show the reader involves the EQUATIONS (i.e. differential equations, constraint equations, etc.), not the actual code to implement those equations. If you have a lot of equations, then a diagram or two showing the main objects in the system and their main interactions is helpful. For example, you might diagram a neural network and the sensors and wheels of the robot that the network controls. In addition, drawings of the different neuron layers and their interconnections often conveys the essential essence of a neural network. Equations for the nodes in each layer, along with those of the synaptic learning rules, complete the picture such that an eager reader could begin to reimplement your system. Deeper details than this are best relegated to an appendix or a web page.
In many cases, a grader is LESS concerned with the actual results but MORE focused on your ability to explain those results. Good analysis can easily be a 1-2 letter-grade difference in a thesis. This is where you can show your skills as a scientist by setting up proper experiments and formally assessing their outcome. For example, if using an evolutionary algorithm to solve a problem, the results of a single run have absolutely no statistical significance, whereas 20-50 runs carry some statistical weight.
In addition to quantitative (i.e. statistical) analyses, qualitative explanations are also important. In many cases, you will lack conclusive, quantitative proof that A caused B, but you should still give a qualitative argument for the A-B relationship if, in your opinion, it helps to explain your results.
It is okay to add a few anecdotal descriptions in the conclusion, such as how much trouble you had working with a robot or how much you learned by implementing a recurrent network from scratch, but these should ONLY be side commentaries, not the main theme. The thesis document is typically NOT intended as a description of the PROCESS that you've been through. It should only contain the key RESULTS of that process, in terms of what you have a) learned about your field, and b) contributed to your field.
The REQUIRED SECTIONS of the thesis ARE strict. The thesis is a scholarly piece of work in computer science, and it must thereby abide by the standards of the field. A failure to include each of the 6 chapters listed above will almost inevitably result in a lower grade. Our reviewers EXPECT to see them. ANY significant deviation from this general section organization should be discussed with your advisor prior to writing. Failure to do so could have major negative consequences, even though the advisor does not grade your thesis. There are myriad opportunities to be creative when writing a master's thesis, but the overall structure is not one of them.
In general, a good thesis process covers each of the 3 phases above, with approximately equal time devoted to each. A good thesis includes the sections listed above.
The very best theses do find interesting open questions in contemporary research and make some real headway in solving them, but these theses are quite rare. Please remember that not all research is about success and breakthroughs. We all have that ambition, but the odds are not in our favor. Most research projects end in a wimper, not a bang. You begin with high hopes for your system, but in the end, it may only do 20% of what you dreamed. The important thing is not to quit and call the whole project a failure, but to analyze the results and write them up. Your work can then help others who are interested in a similar problem, both by telling them what to do and what NOT to do. That's research - both the successes and the failures.
I hope that every master's thesis can turn into a publishable piece of research, but this is also overly ambitious. Not every project will make a new contribution to the field, and that is fine. This is not a PhD thesis!! My main concern is that each master students "comes up to speed" in an area of research. This happens via reading (lots of it!!!) and (in my fields of study - evolutionary computation and artificial life) implementation.
If you are very astute, you will find a "hole" in contemporary research; and if you are very clever, you will design a system to fill that void. Finding the holes is non-trivial, but good places to start are the "future work" and "discussion" sections of research papers. Also, review articles on particular specialized topics are often extremely useful in this regard, since they tend to give general overviews of both the contributions and weaknesses in a field.
Many masters students do not find any significant holes that they are able to fill. Again, that is okay. It is not a requirement of the masters degree, although it may be the difference between an "A" and a "B" or "C" grade. In general, you do not need to fill a significant hole to get an "A", but you have to a) identify the hole, b) make a serious, well-justified attempt to fill it, and c) analyze very thoroughly the successes and failures of your attempt.
Although the publication of a masters thesis is a possibility, this is the exception, not the rule. The publication itself will be its own additional reward. So please do not get too hung up on the publishing. Just focus on your chosen topic. Learn as much as you can, and in the course of doing so, you may stumble onto something (a hole) that you can turn into something very significant. "Chance favors the well-prepared mind", as Louis Pasteur said. So do that prepararation as thoroughly as possible, and maybe lightning will strike! But if it does not, you can still get a good grade by doing well-justified work and writing an interesting report that displays your new-found knowledge. If you do "hit it big" and get a publication as part of your masters work, do not assume that IDI will then automatically give you an "A" on your masters thesis. In the past, several master students have gotten publications but did not get a thesis "A". The thesis is judged completely independently of any conference or journal paper
A masters thesis in AI is not simply a discussion about something that interests you. It must contain the basic elements of the scientific method: hypothesis, methods, results, analysis, etc. You are a scientist exploring a question, and as a COMPUTER scientist, your natural tool of choice is the computer program.
Remember, the computer program in and of itself is not the RESULT. Although this may be the case in certain engineering disciplines, in AI, the program is the TOOL for PRODUCING the results. So never conclude that since a) you've pointed out an interesting research issue and b) you've written a bug-free program, then c) you are done. The behavior of your system must be analyzed with respect to the research goals. Failure to report and analyze these results and to explicitly link them to your research goals is often the difference between a good (A-B) and a bad (D-F) grade.
Many very good projects end up with a B or C grade due to a variety of problems. The main ones are discussed below.
Time management is clearly a big problem for many students. This typically does not rear its ugly head until a few weeks before the due date. By then, it's normally too late to salvage a top grade. Students fail to recognize that writing up takes a long time and that it can involve time-consuming returns to coding.
A related problem occurs when students SEE an issue that, in a perfect world, would only take a day or 2 (or hour or 2) of recoding to correct, but they simply have no time left to go there. So the thesis includes hints as to how their system could be improved. If those hints are simple things, then a good evaluator will immediately ask, "Why the #&%! didn't (s)he do that?" The answer is clearly "time constraints", but, unfortunately, that is rarely considered a valid excuse (for a project with such long duration). So the letter grade takes a hit. The student MAY opt to not mention this issue in the report, but if the evaluator sees it anyway, then the consequences can be even worse. Either way, the student loses. The moral is to start writing up several months before the deadline such that, when you REALLY begin to think about your system and results (as a normal side-effect of the writing process), the problems that you uncover will have a decent chance of being rectified.
When it comes to DIFFICULT improvements of the system, these are easily listed as "future work", which an evaluator will normally view as a purely positive aspect of your thesis. Namely, it indicates that you are very aware of your systems strengths and weaknesses and have given deep thought to how you would improve the system if you had a few more MONTHS or YEARS to work on it.
Some students do a lot of literature investigation and write extensive "Related Work" chapters, but they do not RELATE their own work to the literature. This weakness is easily detected in the conclusion chapter, where some students merely reiterate their system and its results, without ever connecting to the key research issues (or even their own research questions) or the contributions of others.
This problem is deemed highly problematic by most evaluators for a very simple reason: most masters projects do not produce earth-shattering results but, rather, represent "first steps" into a field by the student. So since the final result will probably not be something extremely useful for the student in their future career, the evaluator hopes that, at least, the student has learned considerable concepts and techniques from a research field, which they may then be able to RELATE to problems that they will encounter in their careers. If they cannot RELATE them to something that they have worked on for the past year, then it's a bad sign!
A really bad thesis is one that lacks any connection to significant research issues. You may hack up an awesome program with fancy 3-d graphics and a complete "fan base" of users, but if you can't describe your system as a valid attempt to solve a problem that RESEARCHERS in AI consider noteworthy, then you'll be looking at a D or worse for a grade. The differences between research and development are real and extremely significant for an academic piece of work. Don't lose sight of them in the midst of those months of programming.
For a description of NTNU's guidelines for evaluating masters theses, look here .