Thursday, August 1, 2019

This blog has moved!

To anyone who reads this, you're probably looking for some newer posts. That's cool, and I love that you're reading! However, I've long moved my site away from blogger. There were too many different challenges for someone who wants to add in custom code.

Honestly, it really comes down to the fact that I wrote a desktop in a browser, and got it working well enough to host the whole site.

You all should go to http://CannonContraption.github.io/ to see some newer posts.

Cheers!

--CannonContraption

Friday, June 8, 2018

Lisp- Recursive Glory

Recently I've been toying a lot with the idea of doing more lisp programming to solve some of the tasks I generally would do with Python or BASH. Lisp often gets written off as old or weird, and while it is old and weird, it's certainly not past its usefulness. Here are a few of the advantages I've seen just from messing with it the past few months:

Simplicity

Probably the most obvious advantage of a language like Lisp is that it has few core constructs. To illustrate, here is the basic syntax of any Lisp statement:


(command argument argument (command2 argument)) 

So there's a few things in this which look odd to the experienced C-esque or BASH-esque familiar. First and foremost statements are all parenthetical. The very first character is a parenthesis. While this has earned the language the nickname "Lots of Inconvenient Superfluous Parentheses" in the past, it's also its core strength.

Lisp got its name from this very unusual syntax. The language was originally an abbreviation of "LISt Processing". Upon closer examination, it becomes clear that everything in that statement above could be considered part of a list. Indeed, this is exactly how Lisp interprets it and is a core part of its simple nature.

So with all that said, let's try and parse what's actually happening here. Just like in mathematics, we start with the inner set of parentheses. Inside those are the words 'command2 argument'. What happens here is that the Lisp interpreter will execute the function 'command2' with the argument 'argument'.

The only other real language structure that you would probably need to be aware of is the single quote. Here's another example:


'("Joe" "Jill" "Jonathan" "Jacob" "Jenna")

Yet again, quite simple looking. You'll notice first and foremost however, that we started with a string. If this worked like a normal list, that wouldn't work. Instead, what this is is a list of data. It is indeed possible to actually try and run this later (which is part of the power of lisp) but the single quote prevents any single element (be it a list or an atom- a single element of a list) from being evaluated. This means if you have a variable chocolate, and you wanted to pass the word chocolate to the function instead of the variable (similar to pass-by-reference if implemented correctly) you would run


(command 'chocolate) 

and instead of getting the value of chocolate, you would in fact get the word chocolate.

That's it. Those are the two real main structures to keep in mind.

Recursive Glory

Part of the draw of Lisp is the simplicity I mentioned above. While this is nice from a structure perspective, Lisp is also really useful because it plays well with recursion. In fact, a well written Lisp program may never need to explicitly declare a variable. Not once. Let's take the example of summing the values 1 to n. Mathematically, you could write this as the equation you see to the left of this text (Unfortunately, I couldn't get it to format correctly inline). In lisp, this is quite simple to express:


(defun sum (n)
 (if (= n 1)
  1
  (+ n (sum (- n 1))))) 

For the curious, the syntax to set a variable is (setq varname value) and if you want to define a global parameter, (defparameter varname value). You'll note neither is used here at all.

So here's what's happening:

First you hand it n. Let's say that's 3 in this case. We know, 3+2+1 = 6 so this is easy to test.

Next, it checks to see if n is 1. If so, it just simply evaluates 1. The last thing you evaluate is the return of your function, and because of this we return 1.

If it's not 1, it adds n to whatever (sum (- n 1)) is.

In this case, the computer will actually end up adding things like this:
((1) + 2) + 3 = 6
I've put parentheses around each run of the function. If we were to write this in lisp's preferred syntax:
(+ (+ 1 2) 3) = 6

You can see that this syntax perfectly represents our order of operations for any n calculations. For those notationally inclined who don't already know, this is commonly called prefix notation. It is also sometimes called "Polish notation" because the first person to popularly theorize its usefulness was Polish. Those who have used HP calculators will likely know about Reverse Polish Notation (RPN), or postfix, which is simply the same notation with the operator at the end.

Let's take one more look at this function, this time in plain old C for those who know that best:

int sum(
   int n)
{
   if(n == 1)
     return 1;
   return n+sum(n-1);
}


As you are probably aware, we really don't code C this way much at all. The more common way to do something like this is a for loop:

int sum(
   int n)
{
   int sum = 0;
   for(
     int i = 1;
     i <= n;
     i++)
   {
     sum += i;
   }
   return sum;
}



Which while better formatted in C than the recursive solution, and certainly more common, this solution takes a lot more code to write. It also requires the use of more structures. Furthermore, in both of these C programs we are forced to declare what we are returning. This means declaring a local variable as well.

And one final note on the syntax here, let's imagine this code without the line breaks. It should become clear very quickly which one is easier to read and understand in this format:

(defun sum (n) (if (= n 1) 1 (+ n (sum (- n 1)))))
int sum(int n){if(n == 1)return 1;return n+sum(n-1);}
int sum(int n){int sum = 0;for(int i = 1;i <= n;i++){sum += i;}return sum;}

I don't think anyone will argue this is the right way to code, but nonetheless, based on character count alone the Lisp version wins. Looking closer though, you can clearly see what the Lisp program is doing at any point along the line about as well as you might be able to with a mathematical formula. It's well compartmentalized, and all you have to see to know what executes first is where the parentheses are.

In the C programs (especially the second one) inference of logical notation can't take you very far, in fact you have to know the syntax of C and you also have to be knowledgeable in its specific assignment-and-addition operators to read it. In the Lisp version, however, all you really need to know is that defun is define a function, and it takes the argument n, and an if statement is the conditional, what to do if true, then what to do if false. Everything else virtually explains itself.

Conclusion

In a world where programmer time is more expensive than CPU time, Lisp makes sense.

It's clear that I like Lisp. It's a capable language, which lends itself well to reading and writing programs logically. It doesn't exactly seem easy at first, but after you understand the basic structure once, it's not a far leap to begin extending the language - in the language. Even C and C++, known for their flexibility, have to resort to a messy preprocessor syntax to really be extended in any way at all, meaningful or otherwise. Lisp allows for one program structure to rule them all.

So here's a question that I get asked a fair amount when I start talking about Lisp:
'Why oh why would you ever want to use it?'

My answer is usually,
'It's clean code for any purpose. You can write the language in the language; you can write really complex stuff in ten minutes, and explain it to someone who's never seen code in five.'

While that doesn't justify things or go into detail like this post does, it usually will get someone to do one of two things:

(or (call me crazy) (try it for themselves))

Sunday, September 3, 2017

Code Style, and How To Write for Reading

Recently I was browsing through YouTube, and it kept suggesting this video about programming mistakes, with one of those paper-cutout like CG thumbnails. My first impression was that it was one of those "let me show you how to code" videos, like so many of YouTube's suggestions to me are. The fact of the matter is, while YouTube tried to get me to watch beginning coder videos, like normal, they cycled through my feed pretty quickly, this one stuck around for a few days.

I don't pretend to know the algorithm YouTube uses to display this stuff, but somehow it figured I would like that video. Intrigued, I clicked it and braced myself for what would surely be someone explaining why using single letters as variable names is bad.

That's not what it was.

One of the odd things I like to watch there is lectures. It doesn't have to be computer science related, but I like watching people talk about something they're passionate about. Every time I learn something new about the subject which I hadn't previously even thought about.

This video, despite it's code-school thumbnail, was one of those videos.

Up until I watched it, I was a hard line fan of Stroustrup indentation. Functions like this one below were nice to me, I was used to them, and they were rather standard.



int do_a_thing(int this, string that, bool the_other_thing){
    if(the_other_thing){
        cout<<this<<endl;
    }
    else{
        cout<<that<<endl;
    }
    for(int i = 0; i<this; i++){
        do_something_with_i(i, this, that);
    }
    return 0;
}

I'm not going to get too deep into the topics in the video, I'll link to it at the end, but my takeaway was basically a whole bunch of stuff I had figured out before, but not really internalized. Here's the shortlist:
  • Comments get out of date fast, often conveying wrong information (In the words of the presenter, comments are lies waiting to happen)
  • It's OK to put a line break in an argument list
  • Variable names should be human-readable, not Java-human-readable or abbreviated to the point of meaninglessness
    • This means no networkedAntiTheftVisualRecorder, just use securityCamera
    • Something out of my code to this effect: menubhoverout is bad, menuButtonMouseLeave is better
  • Layout matters, and consitent predictable layout makes for readable, debuggable, sustainable code
These aren't exactly the points that were made, but again, not trying to repeat what was in the video in its entirety here.

This prompted me to look at my largest, and arguably most worked on free time project, the FSU CS Club site. I've discussed a little about the site in my first post, and here I'm going to show the change in code style since I watched the video.



function movewindow(currentwindow, increasex, increasey){
    //client window bounaries: get the current dimensions of a window
    var cwbounds = currentwindow.toplevel.getBoundingClientRect();
    //screen boundaries: get the current dimensions of the screen
    var scbounds = document.body.getBoundingClientRect();
    //new X position (from top left corner)
    var newx = cwbounds.left + increasex;
    //new Y position (from top left corner)
    var newy = cwbounds.top + increasey;
    //now we make sure we're not running off the screen in the horizontal direction
    if(newx>0 && cwbounds.right+increasex < scbounds.right){
        currentwindow.toplevel.style.left = newx + "px";
    }
    //and try to make sure we don't run off the screen in the vertical direction
    //though the code for the bottom doesn't work right, not sure why.
    if(newy>0 && cwbounds.bottom + increasey < scbounds.bottom){
        currentwindow.toplevel.style.top = newy + "px";
    }
}

Note that most of these comments are obvious, and a number of them would be with better variable names. Also note that the code is largely horizontal, and can make for some long lines.

Now the new code:



function movewindow(
    currentwindow,
    increasex,
    increasey)
{
    var currentWindowBounds = currentwindow.toplevel.getBoundingClientRect();
    var screenBounds = document.body.getBoundingClientRect();
    var newx =
        currentWindowBounds.left +
        increasex;
    var newy =
        currentWindowBounds.top +
        increasey;
    if(
        newx>0 &&
            currentWindowBounds.right + increasex < screenBounds.right)
    {
        currentwindow.toplevel.style.left =
            newx +
            "px";
    }
    if(
        newy>0 &&
            currentWindowBounds.bottom + increasey < screenBounds.bottom)
    {
        currentwindow.toplevel.style.top =
            newy +
            "px";
    }
}

When trying to debug the second version of this otherwise identical code, it's much easier to find specific variables, troubleshoot problems, and even understand to an extent. Also note that as parts of this program expand, the general format is not broken, and horizontal scrolling is not an issue. Switching to this style virtually eliminated >80 character lines, making it readable alongside not one more document, but two more on my 1080p monitor. With the old style, some functions would become unreadable if they didn't have at least 2/3 of the screen to display their enormous lines on.


And this comes to the final part of this post: why any of this matters.

I've seen a lot of lectures where someone will recommend something only for it to turn out to not be very useful. There are plenty of people who use some K&R derived indentation scheme like I was, and to good effect, it's a solid style. This time, however, the change in something as trivial as style has made the code arguably neater and easier to not only read, but expand upon down the line without breaking the readability of the code.

I've been using GNU Emacs for my coding of late, and I do want to mention that it makes all of this really easy. A lot of my code had bad indentation, or hadn't been changed when a new level of code was inserted above another existing level, and it made for some trouble. When formatting all of this, if a mistake crops up, Emacs is intelligent enough that simply selecting the region where things went wrong and typing C-M-\ will solve indentation problems, and even go so far as to place the braces on the correct indentation level to separate code from parameter lists properly. Emacs will also let you align arguments to a function right at the end of the function name, where the list would run horizontally otherwise in Stroustrup style.

Here's the original code sample in the new format:


int do_a_thing(
    int this,
    string that,
    bool the_other_thing)
{
    if(the_other_thing)
    {
        cout<<this&lt<endl;
    }
    else
    {
        cout<<that<<endl;
    }
    for(
        int countsUpToThis = 0;
        countsUpToThis<this;
        countsUpToThis++)
    {
        do_something_with_i(
            countsUpToThis,
            this,
            that);
    }
    return 0;
}
Much neater, right?

Original lecture video (Edit: Appears to be broken?)

Thursday, June 22, 2017

ANSI Color Block Graphics

Before reading, please note that anything surrounded by 'm/ and /' is to be interpreted as regex. If you are not familiar with regex, or REGular EXpressions, please  visit this CodeProject page. If you know a little, but don't know the syntax I'm using, visit the PerlDoc page for regex.

Introduction

For those who know how the world of sh and POSIX-like systems works, the idea of using escapes to color text is probably a very familiar one. Using that to create graphics, however, may not be so familiar.

The ANSI Escape(s)

In my post on how to make BASH prompt you with return and git information, I used the 'm/\\033[7;#m/' sequence to turn the background various colors. This works for the most part, but it caused problems during my testing when trying to overlay text. Because of this, I started using a slightly different set of color codes. In the case of text colors, the 'm/\\033[3#m/' sequence is how the color is set. In this case the 'm/3#/' portion of that sequence is important. Numbers in the 30's from 30-37 indicate colors. This is what is called "foreground colors".

To use foreground colors to color a background, one needs only change our 'm/\\033[3#m/' pattern to 'm/\\033[7;3#m/', adding a 7; before the color code. What this does is it enables a "reverse video" mode. This means that colors are kept at regular intensity (so no 1; prefix, we have a 7; there) and foreground colors populate the background and vice-versa.

The other method of coloring the background is to change the color sequence entirely. Where in the last example, the color codes have all been 'm/3#/', if we swap this pattern for 'm/4#/' where the number in question is the same set of values as the 'm/3#/' pattern, we end up with background colors changing. This has the benefit of allowing the terminal emulator to judge which colors are appropriate for the foreground without setting them, so if you have a black background terminal and set a black character, the text on top will still be visible. If the reverse-video modifier is used, the text takes the background color and therefore is invisible.

Escaping the Graphics

So far what I've been talking about has been about color escape sequences, and that's nothing new here. Where things get interesting is when you apply these sequences to try and make graphics.

I have been working on a small text adventure game. It has been acting as a passtime and a time waster at that, and it may not even ever be finished. However, I didn't want to include any graphics libraries of any sort during the creation of the game. This means that, since I'm using my language of choice, C++, there is no sort of imagery whatsoever without color escapes or something similar. Here is where the interesting bits are.

If you assume that wherever the program will be run, it will be run in a POSIX-like environment such as Linux (the only OS any of my desktops or laptops run), MacOS, or Cygwin, you can assume that these escapes will work. In order to make imagery and make the game more interesting, all one needs do is set the color escapes, and type a space for each character they want that color. For example, if I wanted to print a small French flag, I could do so with the following code:




\033[41m   \033[47m   \033[44m   \033[m\n
\033[41m   \033[47m   \033[44m   \033[m\n
\033[41m   \033[47m   \033[44m   \033[m\n



All that I would need to do from here to put this in my program is remove the line breaks (which I added to illustrate the pattern) and then paste this sequence into a cout<< line. The result would look something like this:




#include
int main(){
  std::cout<<"\033[41m   \033[47m   \033[44m   \033[m\n\033[41m   \033[47m   \033[44m   \033[m\n\033[41m   \033[47m   \033[44m   \033[m\n";
  return 0;
}


While this looks ugly from a programmer's perspective, and probably would be much cleaner (not to mention better code) if instead of one cout I had used three, one per line, and instead of using \n to break the line I had used endl, but for the sake of my own sanity, doing that over twenty or more lines seemed impractical, and so I used the method I have so far illustrated.

How to Save your Sanity in the Process

That aside, coding large graphics in this way is tedious. When the end goal is to create a complicated image or graphic show up on screen, inputting escapes by hand gets old fast.

In my case, after less than ten seconds of thought, I got to work developing a small Python script which would replace individual characters with colored spaces. My first design was to simply make something that would replace a single character (g for green, for example) with the appropriate escape code and a space. Immediately, I knew this was the wrong approach. Assuming that the same color is repeated more than once, the characters required to reissue the escape for every repeated space would be massive, and pasting large amounts of program output into a document can sometimes cause issues with text editors which cannot handle long lines. While this was a design issue with the easiest solution, I knew of a better way.

My second and final design checks for repeated characters by storing whatever the last used color escape was, and only outputting a new one if the current and previous escapes did not match. This means that if I were to input a line of black characters, it would issue the black color, fill the line with spaces, and move on until it hits a different color.

Here is a pattern I am using in the game I mentioned:





rrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlllllllllllllllllllllbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrlllmmrrrrmmmmllllllllllllllllllllbbbbbbbbbbbbrrrrmmmmrrrrmlllrrrrmmmm
mmmmrrrrlywwwwllmmmmrrrrlllllllllllllllllllbbbbbbbbbbbbbmmmmrrrrmmlywwwwllmmrrrr
rrrrmmmlywwwwwwllrrrmmmmllllllllllllllllllbbbbbbbbbbbbbbrrrrmmmmrlywwwwwwllrmmmm
mmmmrrrlywwwwwwllmmmrrrrlllllllllllllllllbbbbbbbbbbbbbbbmmmmrrrrmlywwwwwwllmrrrr
rrrrmmmllllllllllrrrmmmmllllllllllllllllbbbbbbbbbbbbbbbbrrrrmmmmrllllllllllrmmmm
mmmmrrrlywwwwwwllmmmrrrrlllllllllllllllbbbbbbbbbbbbbbbbbmmmmrrrrmlywwwwwwllmrrrr
rrrrmmmmlywwwwllrrrrmmmmllllllllllllllbbbbbbbbbbbbbbbbbbrrrrmmmmrrlywwwwllrrmmmm
mmmmrrrrmlywwllrmmmmrrrrlllllllllllllbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmlywwllmmmrrrr
rrrrmmmmrrlyllmmrrrrmmmmllllllllllllbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrlyllrrrrmmmm
mmmmrrrrmmmllrrrmmmmrrrrlllllllllllbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrllrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmllllllllllbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlllllllllbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmllllllllbbbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlllllllbbbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmllllllbbbbbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlllllbbbbbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmllllbbbbbbbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlllbbbbbbbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmllbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrlbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr
rrrrmmmmrrrrmmmmrrrrmmmmbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbrrrrmmmmrrrrmmmmrrrrmmmm
mmmmrrrrmmmmrrrrmmmmrrrrbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbmmmmrrrrmmmmrrrrmmmmrrrr


Even in pattern form, it is clear that there are distinct patterns in the text. In order to turn this into escape sequences, I used the following Python script:




#!/usr/bin/env python3

import sys;
"""
l = black char(30)
r = red char (31)
g = green char (32)
y = yello char (33)
b = blue char (34)
m = magenta char (35)
c = cyan char (36)
w = white char (37)
"""
filename = open(sys.argv[1], "r");
lastcode = 30;
code = 30;
for line in filename:
  for cchar in line:
    lastcode = code;
    if cchar == '\n':
      sys.stdout.write("\\n");
      continue;
    elif cchar == 'l': code = 40;
    elif cchar == 'r': code = 41;
    elif cchar == 'g': code = 42;
    elif cchar == 'y': code = 43;
    elif cchar == 'b': code = 44;
    elif cchar == 'm': code = 45;
    elif cchar == 'c': code = 46;
    elif cchar == 'w': code = 47;
    if code == lastcode:
      sys.stdout.write(" ");
    else:
      sys.stdout.write("\\033["+str(code)+"m ");
sys.stdout.write("\\033[m\n");


This then prints to stdout (the terminal) the exact string I should paste into my program in order to show the graphic I patterned above.

Let's say I wanted to test this first, just to see what it would look like in color. If we name the above color map doorway.cmap, we can run a single line in Bash to print it to the screen:




echo -ne "`./colormap.py doorway.cmap`"


This is the result of that image map:
The resulting cmap, as printed by echo on an 80x25 character terminal
The result is color graphics. Escape sequences can't do a full color palette, having only seven choices including black and white, and my monospace font doesn't have half-height characters to split widths and heights to gain more resolution. So, given that background colors and spaces are the only tools used, images like this are possible using this procedure.

In the end, the result is a graphics-enriched text based game, for which I can generate graphics easily from a text editor. My first few images I first drew in a small Gimp canvas, but before too long I realized that it was almost as quick to forego gimp entirely, and since Gimp can't zoom in one dimension and not the other, the resulting images there always looked half the height of the actual end product.

And that's how you make color graphics using ANSI escape sequences and spaces.

Saturday, February 25, 2017

Web Programming a Desktop with Javascript

One of the things which most people in web programming have to deal with is the problem of interactive content. The oldest methods for solving this were through the use of Java applets, and Flash content. There are other ways to accomplish this too, with software such as QuickTime, Real Player, VLC Media Player, and many more. In today's world, however, the use of most of these is an out-of-date concept. Most modern web applications are written using the features of HTML5, which fills the need to have built in media support, JavaScript, which fills the space of dynamic content handling, CSS, which handles layout and style separate from HTML, allowing for modular, even animated styling support, and AJAX, which is a small subset of JavaScript which is useful for fetching and sending data without reloading a page, filling the need for single-page dynamically loaded content.

Why I care

Last October, I joined the Framingham State Computer Science Club. It is a decision which I am very glad I made, for one simple reason. I was sitting at a table with three excellent individuals, and we decided we should make a website for the club. This is where my JS adventure began.

Our first several drafts of what the site should be generally boiled down to conveying information to club members on meeting times, discussion topics, and other useful information. It was always meant to be something which would be useful first and foremost to club members, and a point of interest for those who might join. Eventually, we settled on the idea of making the entire site work like a large blog site, where members of the club can post interesting posts of the same nature as this blog. After running through several interface ideas, the one that stuck the best is writing a desktop-paradigm interface in a browser, using the new technologies I mentioned in the introduction.

Assembling the Tools

One of my first instincts when writing an interface of this sort is to take an objective look at exactly what needs to be done. My first thoughts in these matters were that we needed to work first on which toolkit we would use. I looked at JQuery first, because I had used it for some projects before. The first thing that hit me was version 3 in fact tried hardest to be compatible with Internet Explorer, and had a note about compatibility with more modern browsers at something along the lines of latest -1 version. This was not acceptable at all to me, because none of the people I know who would be using this site would be using IE. While it was likely that the features would work on many older browsers, JQuery didn't seem to actually add much value to a project like this, and was a relatively large JavaScript library, so I quickly moved away from that.

I mentioned in my last post a friend of mine, whom I actually met because of this project. He directed me to another person's personal site, that of Samy Kamkar, and mentioned that he had used extjs. Upon closer examination, I found that he had used some clever tricks to stop the casual observer from finding his source code, labeling it as a challenge for those who really want to learn web. While I respect this, I would generally prefer to go into a project with a new library with an example, and this site wasn't going to make that easy.

This brought me to the conclusion that I really should just learn enough JavaScript to implement this without any third-party libraries. That way, the code will run quicker, smoother, and without extra overhead from functions and programming I would not be using. As to that last point, the library is not any smaller than what it would probably be with someone else's library simply because I added so many comments to the code in order to facilitate its repurposing and reuse. Unlike Samy Kamkar, I would like it to be easy for some casual onlooker to fire up the CS club page and see how it works.

Window Management in a Browser

The first challenge I had to face was how exactly I would get windows to exist in an HTML space. It was clear that it was possible, others had done it before. I had already assembled a taskbar and rudimentary start menu, those were simple. All that took was knowledge of CSS3 transitions and the z-index property. After much thought on the matter, I started investigating how JavaScript handles mouse positioning. I found that it deals in coordinates relative to the top left corner of the page, much like most graphics libraries and engines. This made it simple. All I had to do was record where I clicked the mouse down, where I released the mouse button, and then take the difference of the two in order to figure out how much to move the window. My first prototype window toolkit used this method. The biggest problem with this was that I couldn't see where I was moving the window to while I was doing it. I could see where I started, and where the window ended up, but no in-between.

After much tweaking, I found that the best way to handle the issue of window placement while moving was to actually make the browser update the window on the onMouseMove event. This would mean that every time the browser registered the mouse moving, there would be some JavaScript code to execute. My first thought was that this would make the entire interface unbelievably slow, but that expectation has largely been overshadowed by the fact that I have tried this on several computers, including a Dell Inspiron 6000 from 2003, and none of them have had a problem with it so far. As it turns out mouse position polling does not take that much work to accomplish. After hacking away at it for a little while, I managed to get a system which would detect when the user was dragging the window, and update automatically to reflect this. Window management was mostly done after that, as it was a simple matter to implement the JavaScript code to add and remove HTML elements with no ID and simply track them through JavaScript. This proved to be a stable and surprisingly quick solution which is still in use now.

Window Components and Layout

Another challenge I faced early on, perhaps even earlier than getting windows to move, was how to lay out a window so it has the same components that everyone and their uncle recognize from using Windows. The main components that I would need to add were:

  • Titlebar
  • Window title text
  • Window content area
  • Close button
  • (Time permitting) Minimize and Maximize buttons

I decided to stick to just implementing the close button at first, as the window manager would need to mature significantly before I could start working on the other parts.

The first design I had in mind was a master toplevel div, which would float in a position:absolute; style and be the thing which would hold all of the components of the window. Each component would sit in this toplevel div, and would actually be what showed content. The toplevel itself would provide a border around the whole window.

That didn't work out quite like that. One of the oddities about HTML is how div elements work. You can nest div elements inside each other without trouble, but things become tricky as soon as you want one div to start before a nested div and end after it in vertical space. It simply doesn't work this way. What I ended up doing was I defined the height of the toplevel div to be about 22 pixels or so (I can't recall the exact value, but the code is public so the interested reader can look it up), and then nesting all the child elements inside that div like I had intended. The toplevel served as the background for the titlebar, and I simply made the window content area show up white. Window buttons would show up on the right-hand side like in Windows and KDE (the desktop I use almost everywhere), and the title text would be centered in the window. The only parameter that would need to be specified when creating a window in terms of dimensions would be how wide the window should be, that way it can expand in height dynamically to fit the content and I don't have to worry about making the arithmetic work right even if fonts change depending on the browser and operating system. Being a Linux user, I care about this stuff as many of my friends and fellow classmates use a completely different font set than I even have access to. The only step after determining all this was to actually connect signals and make the window react predictably. Since this was a simple matter, and the details of how I did it are in the code itself, I will leave that out of this post.

Determining How to Make Content in a Window

Another interesting challenge was how I should then make the window available to end-developers to use. At the time I had (perhaps wisely) decided I wanted the windowing code to be a discrete component which could later be taken out of context and used in a different site, and I didn't want the task of actually populating windows to be too much of a task.

My first solution was to simply add a section where someone could take flat HTML code and then insert it straight into the window as if it were another page. This was a neat idea, and very good for testing windows, but it raised on serious concern for me: there would be no easy way to ensure consistency of interface across all windows in the same site without some serious talk between members of the project about how to class each element so it matched up with the master stylesheets.

The solution I finally settled on was to keep the option to insert custom code into a window as a way of populating it, but also to add the option to add a single element to a window. This was the dawn of the widgetTools toolkit, which was created as another discrete component to compliment the site and yet still be portable to places outside of the website project.

Drawing Inspiration from GTK+

A large part of how widgetTools works is drawn from my prior experience working with GTK+3 in Python, C, and C++. GTK+ is a wonderfully useful and complete widget toolkit which delivers on the promise of consistent look and feel across applications on a single platform. Unlike competing technology in Java, most notably Swing, it manages to look and feel right in a Linux environment, and in Windows both. I have seen screenshots of it also fitting in almost exactly like Apple's own toolkit in macOS, but I do not own a mac and so I can't corroborate this. The Achilles' Heel of this toolkit is complexity. In order to do some relatively simple things in GTK+, you need to have some knowledge of the data structures and functions that it uses in order to generate the content on the user's screen. While this is all really well documented, and easy to learn if you have the time to figure it out, it is quite complex. Further, while GTK+ has a web backend, it is not built to work like a website, instead it is built to work more like another form of remote desktop, allowing single-application access over a LAN or internet connection. While it does implement window management too, the simple fact of the matter is it wouldn't work for a situation like this because of its single-instance nature.

That being said, I did quite enjoy working with GTK+, and it showed me a lot about what GUI development is and why so many people find it difficult. Some of the things they use in GTK+ are out of date, or workarounds for old problems since solved, but for the most part there is a valid reason for everything they do in that toolkit, which is something I wanted to do myself with mine. So that was what I decided I would do. I had already largely been working with JavaScript in order to write windowTools, so I continued using a similar model for widgetTools. First and foremost, windowTools was written in such a way that it could insert a single HTML DOM object into a window, as a sort of toplevel widget space. For that purpose, I created a simple function to create an empty div in widgetTools called the widget space. The first step in using widgetTools would be to create such an element and set it as the toplevel for the window.

Less Complicated Widget Layout

One thing that I really like about GTK+ is that it covers almost every case. In the case of widgetTools, however, I can't do that so much. A lot of that comes from the fact that HTML has a limited set of primitives to choose from when it comes to creating new UI elements. It is enough to do some really impressive stuff, as anyone who has used Google in recent years can tell you, but it is still not quite as much freedom as the developers of GTK+ had. Nor does it require quite so much work to get a foundation laid. This meant that I could both relax a little because some elements were ready-made for me, but it also meant that I would likely end up doing nothing but wrapping existing elements into JavaScript, which could turn dull quickly.

The main idea behind the widgetTools setup is that in order to place an element, you need only tell the toolkit what its parent should be. This means that in order to get a horizontal split in a single window the developer would have to use a table or something similar, but it also means that if the window did not require fancy layouts like that it could be developed quite quickly without worrying about placement. While I think Gtk_Grid is a nice solution to the problem of layout, it is overkill to the nth degree here, and so it would not be useful. This means that the syntax for creating and positioning a window element could simply be makeButton(widgetSpace, "button", "this is a test button");. That line, if run in widgetTools, would create a standard button (hence the "button" argument) which would automatically be placed in the toplevel widgetSpace below any existing elements (or beside them if there's no block element and the window is wide enough). This system returns the HTML DOM element as a JS object for each item it creates. If, for example, we wanted to put the button in a table data (td), all we would need to do is change the widgetSpace argument to some other object we have created. Assuming our table data is named tdtest, that would make the code to insert a button there makeButton(tdtest, "button", "this is a test button");. This means that in order to determine layout, all you need know is where you wish to place the element, and the browser's existing layout engine can take care of the rest.

What we end up after this is actually relatively powerful:

Minimization, Window Resizing

If I were to write this post in order of how I actually wrote the code, resizing the window would have come into play around the same time as the widget toolkit, but I feel like resize and minimize work closely with each other, so I decided to put them in the same section. Both were at least started after the widget toolkit, although resize was done well before much was done at all in the widget toolkit. For details on how it was put together, reading the commit list on the GitHub repository is where you should go. It is incredibly detailed, so if you're not up for a real commit swim, it may be worth skipping that and just taking my word for it.

Resize

In order to fix the problem of window resizing, I approached the problem from the same angle as moving the window. Take the dimensions of the window when you click the grab handle, then poll the mouse to see where it moves and change the window accordingly. This stops when the user lifts the button. The issue with this was that instead of moving X and Y coordinates alone, I was dealing with adding properties for height so that the user could resize in both directions. This turned out to be less of a problem. When I started working with the idea of resizing based on move commands, I did not know that JavaScript does not take all of the window dimensions into account. It takes the inner dimensions of the window, and the outer dimensions to a point. The trick was not only resizing the toplevel's width, but the height of the content div. Since the dimensions don't quite match between these because I initially wanted a window border, only to settle for the titlebar being a little wider than the window. After a lot of work, I found that if I take the inner dimensions of the window and then only deal with those, the outer dimensions follow suit. All I had to do was modify the style properties for width and height based on my knowledge of how the box model is handled and a little black magic with known pixel count values to make resize work properly, and not exponetially expand into infinity like before I had figured out the missing properties.

Minimize

Like I said, minimize and resize are similar. The first problem I had to solve was adding window buttons to the taskbar, but doing that (and even tracking their location) was quite easy. The real trick was figuring out how to hide the windows themselves. In the end, I decided to add some global variables to represent the position of the window at the start of minimization, and then just change its width to match the taskbar button, and set its X and Y coordinates to match the top corner of the button.

This was the right decision to make in the end largely because it made animating the minimization process really easy. All I had to do there is keep track of how long I wanted the animation to run, and then use that to set CSS transitions so that it would smoothly move from its original location to its spot in the taskbar. Since I had found out during the time when I was writing the move facilities that transitions mess with window moving, I had to make the transitions reset to 0s for all transition properties, but considering what it took to make other functions work, this was a simple enough task.

Conclusion

Making windowing in a browser was a fun project which, in the end, turned out to include some useful components which can simply be placed in other projects to use for basically any purpose. Anyone can view the current CS Club site on GitHub, simply follow this link and you're there!

Monday, February 20, 2017

Making Bash Prompt You with Information

In the world of Linux and Unix, one of the most powerful tools available is the command line. Those of us who are lucky enough to have Bash at our disposal will probably know of many of its varied powers and features. These include the ability to modify and rerun commands in history without retyping or scrolling through text, running simple logic and recursion from within the prompt, and setting custom prompts to display more information than is initially readily available.

It is the last point of these that I want to touch on today.

Introduction to Bash's PS1 variable

If you are familiar with how bash handles PS1 strings and colors, skip ahead to the section about dynamic prompts.

In the past year, I have started exploring the power in how Bash handles the PS1 environment variable. For those of you in the know, PS1 is the string variable responsible for deciding what the prompt will contain. A very basic one may just be PS1='\$ ', which will simply display a dollar sign and a space to signify the prompt is ready for input. A more common arrangement would display the user, the hostname, the working path, and then the dollar sign. This is often represented by the following PS1 string (or something similar): PS1=\u@\h: \w\$ '

This is still really quite a simple prompt, and for those who are familiar with bash it probably doesn't take very much effort to parse. Where things get interesting is when colors are added. Ubuntu, for example, uses green for the \u@\h, making it something similar to PS1='\[\033[1;32m\]\u@\h\[\033[m\]: \[\033[1;34m\]\w \$\[\033[m\] '. Notice that aside from what we had before, we have sections encapsulated in \[ \]. These essentially tell bash that we are no longer outputting printing characters. This helps it align text when a line of input is longer than a single line long. It can figure out where the line break should be, and place it appropriately. Without these delimiters, sometimes instead of getting a new line, overflowed lines will react like \r on its own, or cause other interesting and unintended effects. Inside of these, the escape character (\033) followed by an ANSI color sequence. Here's where I start to get creative.

Making Dynamic Prompts

One of the neat features about bash is that it allows you to set custom functions to streamline a bunch of commands in the current process. In my case, I use this to evaluate the last return code from a program and then use that information when deciding how the PS1 variable should be put together. This works because Bash, unlike so many other shells, re-evaluates its PS1 every time you submit a command. In my case, I like to know whether a program returned 0 or not, and what it returned if it wasn't 0. Since I develop a lot of programs and scripts myself, this is really really handy for debugging things without using a debugger.


setpromptstyle(){
    lastreturn=$?
    if [ $lastreturn = 0 ]; then
        echo -ne "\001\033[7;32m\002"
    else
        echo -ne "\001\033[7;31m\002 $lastreturn "
    fi
}

So from here you can see that things aren't exactly as I described them above. Firstly, Bash doesn't seem to evaluate \[ and \] inside of functions like this, so instead I used \001 and \002, which for those of you who aren't familiar with them are start of header and start of body characters, respectively. Second of all, by using the reverse-video flag in the color sequences, I get an easily recognizable place to stop for when I'm scrolling up to the top of a command's output. For regular users, I usually use green for a zero-return-code, and red for a non-zero one. You'll note that the number itself doesn't show up unless the program actually didn't return zero. This is especially handy when you have a really long working directory.

For a while, I had the following as my PS1 variable:

PS1='$(setpromptstyle)\u@\h\[\033[m\]:\[\033[1;34m\]\w\[\033[1;33m\]\$\[\033[m\] '


Adding Git integration

Just recently, a friend of mine asked if I had ever used powerline, which is a git project for fancy vim/bash/etc. prompt lines. In vim, it clearly showed information like the current git branch. Being a frequent user of git and github, I figured this would be a nice feature to add to my own custom prompt. I didn't want to try powerline myself, in no small part because it would mean changing my current prompt (which I am really happy with) for something else which I'm not as familiar with. Normally I am all for jumping out of one's comfort zone, but this time I felt like it would be more useful for me to create my own variant of git/bash integration.

The first thing I wanted to do was add a counter for uncommitted files. I know this post is about bash, but in this case I didn't want to bother with bash and went straight to perl:

#!/usr/bin/perl
my $gscounter = 0;
for my $line(`git status -s`){
    $gscounter++;
}
print "$gscounter";

I saved this into bin/gitstatuscounter.pl, and put that into a function in bash. Stderr is redirected to /dev/null (2>/dev/null at the end of the command), and it outputs a single number with no line break at the end, perfect for inserting into a prompt.

The next thing I wanted to do was make the current branch show up in the prompt, so it's easy to see which one I'm working on. I've messed up which branch I'm working with more times than I'd care to admit, and so this is where the real usefulness of this project comes in. As it turns out, with a little sed and perl magic, we can make the git branch command output just the current working branch.
git branch | sed -n '/\* /s///p' | perl -pe 'chomp'
This makes git spit out the branch list, and then sed searches it for the line with a * in it, which is the delimiter for the current branch, then perl comes in and removes the trailing line break.

But what if we're not in a repo?

I haven't addressed a major problem with git integration, and that is what happens when we're not in a repo. As it turns out, the fix is quite simple. If git status runs without error, we are in a repo. If it does not return 0, we are not in a repo and shouldn't run the git integration code. The git integration code can now be shoved into a single bash function like so:

getgitbranch(){
    git status > /dev/null 2>&1
    if [ $? == 0 ]; then
        echo -ne ' \001\033[7;37m\002'
        gitstatuscounter.pl 2>/dev/null
        echo -ne '\001\033[7;36m\002 '
        git branch | sed -n '/\* /s///p' | perl -pe 'chomp'
        echo -n ' '
    else
        echo -n ''
    fi
}
I could probably take out that last else and echo -n '' statement, but for a while I was waffling on whether to print something else if we're not in a repo, and that would be the place to do it. For a time, that would print a space there, and it wouldn't be padded so much.

Applying our "wizdom"

When I looked at the powerline screenshots from vim, one thing struck me- the entire thing used the reverse-video flag (the 7 in my escape sequences). After giving it about five second of thought I figured this would only make the prompt more visible since I can look anywhere across a line to see where I am. One of the things about my directory tree in my home directory on any given machine is that I like to use long folder names and have bash's auto-complete fill in the full name after I typed enough characters to distinguish it from other folders. This means that prompts will often span more than half of my terminal window. If this were all reverse-video, the prompt would be visible indeed.

After a little bit of messing with different options, I settled on this as my final prompt:

PS1='$(setpromptstyle)\u@\h\[\033[m\]:\[\033[7;34m\]\w\[\033[m\]$(getgitbranch)\[\e[7;33m\]\$\[\e[m\] '

And this provides a nice, tight prompt with git integration that hides itself if we're not in a repo (or don't have git installed).

A note on code

Of course, any code I post on this blog is free for anyone to use. If it's posted on here, assume it is licensed under the GPLv2 and (C) James Read unless otherwise specified. Please respect these rules, and if you have something you think would improve any code I post or any solution I have, please just write a comment on that post! If it's really really good, I may even feature it in an edit to the post itself!


Edit: I feel it is probably best if I include a screenshot of what the prompt looks like when all this is said and done. Here are two, one with a git repository, and one outside: