Program The rantings of a lunatic Scientist

Posts marked as L2Program


A note on Programming vs Software Engineering

L2Program Rant

Oh yes, gather round for another classic Joss rant explanation of common development myths and theories.

Today’s story comes to us from another reddit post, this time asking about why there is (in general) a stigma towards sites like Code Academy who aim to make programming excessible and easy for the masses.


I think a big thing here that just needs to be stated is that Programming is not the same as Software Engineering.

Anyone can go online and learn a programming language, the syntax is relatively easy and sites like Code Academy do a pretty good job of teaching that most of the time. But they teach two things, “syntax” and the very very very basics of “program structuring.”

Developers give sites like Code Academy a bad rep because they actively (if not always intentionally) promote the idea that coding is the same as building a piece of software. They say…

Anyone can do <it>! <It's> easy! Come here today and learn <it>! <It> only takes an hour or two…

<It> being to write lines of code in a text editor and run it, but implied to mean you can start the next facebook.


Software Development is barely connected to coding. The code and the language are just how you write it down and give it to the computer. Yes, there is a lot of coding, sometimes millions of lines of it in a large system. But that pales in comparison to what those lines of code represent.

When you design software you build based on design patterns. Not every pattern is perfect for every situation which is why you need to know a lot of them so you can use the best one for each job. Similarly within those patterns you will be using data structures (arrays, hash tables, linked lists, databases, ect). Again, not one of them works perfectly for every use case. You need to know how they work and which is the best to use for each situation.

Anyone can learn to write int[] a = new int[10]; or HashMap<String, Object> t = new HashMap<>(); in five minutes. It takes a lot longer to learn how they work and when to use them.

That is the difference between “programming” and “software engineering”, and that is why so much of the development community look down on sites like code academy. They paint a false image of the industry and the profession that many of us have devoted many years of our lives and education to.


Now. This is not to say we all know everything and what is best. This is a constantly evolving and changing field and we all are constantly learning and reworking the way we approach the problems presented to us.

I don’t mean to sound discouraging, if I have appeared that way thus far then please have my apologies. That was not my intention. My intent was show why the when the entire world has the ability and accessibility to learn to program and the attitude that they can master web developement and start the next instagram and that it will all be easy because computers are so good these days, that we will still be the same industry we had before they all learned and we will still be the “valued” workforce that we are.

Because you just can’t learn everything we’ve learned by doing tutorials online. Yes we too do online tutorials, watch lecture videos from conferences, go to seminars and master classes.. Yes we do those things too. But we have also been actually doing this work for years, learning from real-life projects, and directly from colleagues who had been doing this far longer than us. Many of us went to university and exclusively studied Computer Science, and algorithms, and software engineering, and project management and lots of other things we use every single day on the job.


In essence, Code Academy and everything it stands for only seeks to invalidate and deprecate everything that gives our experience and our industry value. Even if they did it under the best possible intentions.

A note about GPGPU for the masses

C/C++ GPGPU L2Program Rant

A note about the growing trend towards GPGPU for the masses. This is my response to a Reddit post I saw about a new GPU language, I felt I should copy it here.

As someone who does a lot of development using the GPU a new language is the last thing I want. Programming for the GPU is complicated, it just is, and it should be because what you are running your program on is a very complicated piece of hardware. You have to treat it right and you have to structure your programs and algorithms in a specific way which is not common to other architectures.

All these attempts lately to make GPU programming easy and doable for everyone (the LISP and Haskell libraries come to mind) completely miss the mark. They work under the premise that if you make it easier, everyone can make everything GPU accelerated, and that that will be better. It won’t.

Half the problem with current libraries that make CPU concurrency easier is that people start parallelizing too early. They don’t fully and completely optimize something they already have, they don’t go in and profile it, they don’t notice long system calls and work in a bit of assembly which will reduce the latency. No. Instead they will just chuck some threads in there, because threads make things faster… It’s just not true.

This problem is even more volatile on the GPU. It’s a delicate balance and not every job is suited to the GPU just as not every job is suited to multi threading on the CPU. Giving people (the uninformed people at least) the power of the GPU for every conceivable task is just daft.

When you structure programs for the GPU you need to have full and complete control over everything it does and when it does it. Languages like OpenCL and CUDA may be complicated but your kernels do what they say they do. It’s exactly why writing good C is complicated because you are right on the hardware level with very low abstraction. OpenCL and CUDA don’t try to optimize what you wrote (past a few compile time optimizations which are to be expected) they translate your commands onto the hardware nearly directly. The downside of that is it means you need to fully and completely understand your algorithm and how the hardware will react to each stage of it, the benefit is incredible performance and massively parallel execution.

TL:DR GPU programming is hard for a reason and giving everyone an easy way to do it completely misses the point. It’s like trying to make everyones car into a supercar by handing out nitrous injectors.

And just like that, I have a degree

Dissertation L2Program PhD University

As of this evening I have successfully secured 1st class honours for a BSc in Computer Science. I’d be lying if I said this wasn’t one of the proudest days of my life :)

Maybe the actual graduation ceremony will beat it in a few weeks, assuming I don’t be that guy and trip on the stage!

My final grade for the degree was a happy 79.59, not that it matters too much once it’s inside one of the grade brackets. Least to say I am a little chuffed. The grade for my dissertation has left me happy too with a final grade of 88 in both modules!

This ontop of the recent news that I got PhD funding for next year has left me in a decidedly good mood. Starting this October I will be working towards a PhD in Computer Graphics under my dissertation supervisor Dr. Mark Jones. I’m not entirely sure exactly what I will be researching yet, but I know it’s likely going to be to do with Global Illumination and/or GPU Acceleration; the two areas which I am most interested in!

On the topic of studying Computer Science

L2Program Rant University

So today I saw a post on /r/compsci where the poster was a new Comp Sci student 2 months into his course, having a hard time with programming and not sure how to keep going with his course so he can get a job as a Game Designer. I wrote quite a healthy response to him about how I feel people need to think about CS and I thought I’d mirror it here because it seems to have gone down very well on /r/compsci and because I haven’t posted anything in a few days.

For those interested, here is the post on Reddit. /r/compsci/comments/zdx5v/some_questions_from_a_computer_science_rookie/c63qgsu

Firstly, (IT != CS).


Languages

Something that will become very apparent to you as you study CS is that it isn’t about C#, C++, or any one programming language. Yes you may have modules on certain languages but they are merely an ends to a means. That is to say, they don’t teach you C# so you learn how to program C# they teach it to you so you learn how to structure a hierarchical based Object Oriented (OO) program. Once you understand the theory of how to structure that kind of program you can learn the syntax of any like styled language and immediately run with it. Also bear in mind (rawr) that when they teach you C# they do not expect you to learn every function name in the standard library, they expect you to learn what things you can expect a language or standard library to come with. So that when you move to another like language and the standard library has all different function names you’re okay because you can just search what you need in the Language Documentation online.


Don’t use your degree to learn game development

Computer science is not about game development. However! A lot of the stuff you learn can be applied directly to game development. For instance, you shouldn’t learn “How to write a FPS game.”, you should learn…

  • 3D Vector Mathematics and Trig
  • Algorithm Design
  • Object Oriented programming, and you’ll want to get pretty comfortable with a language so the code is neat and efficient
  • Artificial Intelligence
  • Graphics Programming (Probably using OpenGL)
  • If you have a module on “Game Theory” then by all means take it but bare in mind that Game Theory will be 99% breaking down and modelling very simple games like Coin Flip, Nim, Tower of Hanoi, ect in a very basic mathematical representation and will almost certainly include no programming unless they ask you to model any of those games as a Finite State Machine.

Use your degree to become exceedingly good at building programs

A good Computer Scientist does not know how to do one thing very well, with the mindset that “It’s okay, I’ll get a job doing that very specific thing.” (especially if that thing is Game Development) A good Computer Scientist understands Algorithm Design and Optimization, Data Storage Structures and Sorting, the benefits of Linear and Parallel execution and when to use each, the mathematics behind Formal Logic, Trig, Vectors, the (basic) Psychology of Human Computer Interaction (HCI) and how you can apply that to UI design, Database Design including Relational Algebra and Calculus to model them, …, the list goes on.


The game industry is Cut Throat and immensely competitive

You have likely heard this many times before but your dreams of working for a big game developer like DICE or 2K (just two random ones off the top of my head don’t downvote me because they aren’t your favorites) are probably not going to come to fruition.

Game Studios do not hire people based on their degree. I know, if you look at the recruitment page it says a BSc in CS and some experience working on other titles. This is just to push teenagers who think it would be a cool career to actually go to University and learn what they need to write that kind of software. Games after all, are software, and very complex pieces of software at that.

Truth be told if someone has a good enough programming portfolio and can show off a really impressive demo to a Game Studio they will not care which Uni (if any) you went to. They want good people (the best people) working for them. If you want one of three positions open at a Game Studio that 8000 people have applied for, you need to be the best.


But the best at what?

At a Game Company people do different jobs, they specialize, and then they get very good at what they do. A level designer probably doesn’t know much about how the core engine works, and he doesn’t have to. The Engine Designer probably can’t design a User Interface, and he doesn’t have to. And the UI designer… you get my point.

So pick a subfield of Game Development. If you hate programming and you don’t enjoy it, you are not going to be a professional programmer, or if you are you’re going to be miserable and you’ll write terrible code. There’s a lot to be said for the theory that good developers enjoy coding and writing software, and they do it in their spare time as a hobby despite the exact same thing being their 9-5. Anyway, you don’t like coding, so you aren’t going to be an Engine Programmer. If you like and enjoy Game Theory then maybe you should look into becoming a Level Designer. Designing a map in an FPS and designing a level in a Side Scrolling shoot’em up game use 90% the same skills and theory.


All of that said

You have only been at this for two months. It takes years to become a good and proficient Computer Scientist, it takes years to get to a point where you intuitively know how to layout and structure a piece of software, and even then you’ll make mistakes, and you’ll waste a lot of time redoing things in the “right” way only to have to redo them again.

If you are still worrying about C# and C++ don’t. They take time to learn but you’ll probably find it just clicks one day and you’ll realize that if this does that there a then this will do that and then that will happen, ect, and it will start to make sense.

New Concept Site

L2Program Web Dev

It hardly would have been worth making a RESTful API for the site if I wasn’t going to build multiple interfaces now would it…

Over the last couple of days I have put together a new or alternative front end for L2Program for the purpose of showing additional site designs in my portfolio. Due to how well my RESTful API project turned out I was able to quickly and easily hook up the API to the new front end simply by implementing a new php and javascript interface which added the appropriate HTML to the API data.

This was also an opportunity to improve the API by adding JSONP functionality. JSONP or “Javascript Object Notation – Padding” is a technique used to get around the CrossDomain Resource issue that plagues current AJAX requests. Effectively, AJAX can only occur within the current domain / subdomain. So from my subdomain http://new.l2program.co.uk/I was not even able to AJAX to my api on the main http://l2program.co.ukdomain.

If you think of JSON as being an array of data then JSONP is simply an array of data as the sole parameter of a function.

{colour:”red”,width:100} is JSON

my_callback({colour:”red”,width:100}) is JSONP

One thing you may notice about JSONP is that rather than being some string based object notation JSONP is Javascript! Which is brilliant because as anyone that has included JQuery from the api.google.com or any other CDN will know, Javascript files can be included Asynchronously from any domain! Internal or External!

So in my API I simply added a clause into all the unauthenticated GET methods saying, if a GET parameter ‘callback’ was sent along too, then compute the result as normal but pad it in parentheses and prepend the callback value.

Now in my JQuery I can make a Cross Domain GET request to my API on the main L2Program domain using code like this.

New Rainmeter Skin for my Desktop

L2Program

Currently working on a new skin for my desktop and am really happy with the results so far.

For those that don’t know, Rainmeter is a system for displaying Live Generated Content right on your desktop. It can even be configured so that the elements ignore click events and can’t be moved. Effectively bringing your wallpaper to life!

I started by going on /r/Rainmeter for some inspiration and eventually came across a post which used a rotoscope layer to hide text in the image. In the post the Redditor linked this post on Deviant Art.

From the second the page loaded, I was in love.

Personally I think the original guy cluttered his desktop up a bit too much. I found the original wallpaper at full resolution and set about turning his 1680×150 desktop into one that would work on my 1920×1080.

All of this was pretty easy, the only annoying part was making the rotoscope layer for the foreground sand. In the end to get a good enough edge I had to just manually go along the line very accurately tracing it, and finally adding a 2px feather to hide any imperfections and blend it to the wallpaper.

Next came the coding. Rainmeter is written in .ini files. You organise your skin into a base folder, and subfolders for each individual Meter. You can write more than one .ini file into one of those subfolders but Program will only let you turn one of them on at a time. This allows you to build sever alternate versions of the same Meter, so anyone who downloads it can choose their favourite.

Here’s the ini file for the rotoscope layer. It’s the simplest of all the ones in the skin, I just wanted to show you what the code looked like. You can see the code for all the layers in the Gist.

Only thing about this skin that I cannot distribute, sadly, is that I can’t say where or how I got my hands on HelveticaNeue… me hearties. ARRRR!

New URL schema using URL Rewrite

L2Program Web Dev

Just finished up implementing URL Rewrite for the site. Effectively this means that the normal urls such as

l2program.co.uk/?id=42

Can be disguised as

l2program.co.uk/post/42/Friends_dont_let_friends_use_Internet_Explorer

Why is this good? I mean, one is considerably longer than the other. But who on earth really types in a whole URL manually to access a specific page of a website? I personally can only ever remember doing so while typing long and ugly URL’s off of a lab sheet in High School Chemistry class.

Edit: It should also be mentioned that because the post title is optional in the url…

l2program.co.uk/post/42

l2program.co.uk/post/42/

l2program.co.uk/post/42/any_random_string_of_chars_digits_and_underscores

… Will all work. As long as the /post/ prefix is followed by a digit (and a slash if anything else follows) you’re good to go! The title string is only there so that in an anchor tag it is visible. A user hovering over the link of a search engine crawling will see that string be able to see where the link leads more clearly.

While the new “fake” urls may be longer they benefit from being friendlier. If someone hovers over the link and sees the address preview they will have a much better idea of where their browser is about to be directed. Along with this ‘human’ friendliness the new urls also benefit from being more frendly to search engines. By appending a stripped and simplified version of the article title to the end of the url we have packed the url full of key words relevant to the pages content.

The simplest way of implementing URL Rewrite is to modify or create a .htaccess file in the root folder of your site. Here is the .htaccess file I have written.

There are two Rewrite rules in the .htaccess file. Lets have a look at what they are doing by breaking them apart

  • RewriteRule // This keyword tells Apache we want to define a rule
  • ^post/([0-9]+)(:?\/[a-zA-Z0-9_])?(:?\/.)?$ // If the url passed in matches this regex then Apache will swap it with the next parameter…
  • /index.php?id=$1 // The replace string. $1 represents the part of the previous regex’s capture group. In this case it is the post id

Long story short, that rule matches URL’s of the form /post/42/Friends_dont_let_friends_use_Internet_Explorer and turns them into index.php?id=42 . Similarly, due to the second Rewrite Rule, if a URL matches /post/tag/5/Ray_Tracing it will be converted to index.php?filter=tag&value=5 . Notice how the titles in the url, either of a post or tag, do not matter when the url is converted. This because they are purely there for show. For the benefit of the user or search engine that will be looking at the URL. All that matters is the post id or filter and value.

All this happens behind the scenes though. Your users will never know the truth, and you get to sit there smugly while your code works with incredibly simple urls while displaying beautiful friendly ones.

To help give a complete picture, here is the php function I use to turn an articles title string into a string that can be used in a url. It runs two very simple regex’s on the input string and then returns the result. The first regex simply takes every character or group of characters that are neither letters/numbers/spaces/underscores and removes them by replacing them with an empty string. The second regex finishes the job by replacing any space or group of spaces with a single underscore.

Thus Friends don’t let friends use Internet Explorer… becomes Friends_dont_let_friends_use_Internet_Explorer