Briefcase: a special kind of folder in Microsoft Windows, which synchronizes its contents with another briefcase of the same name when detected. Used to keep volatile documents on floppy disks & USB flash drives without constantly copying and pasting the contents of the whole disk every time it moves from workstation to workstation. Like an rsync daemon. Use case: in the studies of software design & architecture, a storyboard sketch, or supposition about what a user expects or how s/he'll behave. E.g.. This would be the initial node of a flowchart, a branch in main(), GUI dialog panes, or some interaction of user with program. Additionally, analysis of hostile users and newbies ("misuse case"). I am still moribund ("deadlocked," or "sick to death") by a headache that has become cerebral palsy. I have been unable to concentrate on my plans this year. Speaking of contributions to science, you can find my (literally) auriferous portfolio at the magnanimous MediaFire (they're not just for pirates!): https://www.mediafire.com/folder/kr2bjyn1k3gjr/mlptk-recent (Download & read the CARGO-MANIFEST.TXT to ascertain the contents of the archives you seek.) WARNING: ADULTS ONLY. (Explicit sexual content.) Videlicet is still kind of broken. DiffWalk, too, may be faulty. The hyperlink will lead you to a MediaFire directory. I have added new archives (for bandwidth conservationists). The file CARGO-MANIFEST.TXT describes all the contents: _download and read it first_ if you want to know what's in them there archives, which total over one hundred Megabytes, &/or retrieve your preference. What's new: kanamo & transl8 (in MLPTK), Mutate-o-Matic, Videlicet, & DiffWalk. (I said MLPTK was officially dead, but will I let it rest? How about no...) Archivists curating art galleries downloaded from social networks will love Videlicet, which solves the vexing twin problems of automatic attribution and re-configurable data mining. (For those pesky copy protection mechanisms. Videlicet.py easily cuts through Web galleries and markup up to 1/4" thick.) I even threw in the exprimental upnnas: yea, truly this is an epic day. (^- That line alludes to one of the _Juicy Cerebellum_'s author's asides.) The remainder of this briefing describes the salient points of a Python script I wrote to automatically collate issues of my portfolio. Long story short: "diff." Because the large size of the archives I upload has become problematic, I have established a ramshackle mechanism to prepare smaller files for anyone concerned about bandwidth conservation. (MediaFire reports only two Gigabytes since last year, which is no big deal, but I certanly wasn't helping. Also I couldn't think of much else to do.) In case you cared, the usual issues with bandwidth are constriction & latency: to reuse Senator Ted Stevens' "tubes" metaphor, how wide the tube is and how long it is, and either of these can alter an observer's perception of the pressure of fluid forced through the pipe. "When the tubes get full, things can't get through" -- like dead bodies, or the new episode of Veep. Metaphorically one half of this mechanism is a portable diff utility: DiffWalk. The other half is a shell script that identifies changes to the directory tree. Neither is aught remarkable but why don't I talk your ear off about them anyway? Diff is a program similar to cmp, used to compare two files and describe their discrepancies. In common use on Unixlike systems, it is employed to create patch files that require less time to transmit via point-to-point telecommunication than would be needed to transmit the whole file whenever it changed. Because it is so useful an algorithm, and because I've never seen one for Windows (except in the Berkeley Utilities), I made (but didn't test) a portable one in Python. DiffWalk is a walking collater that creates patches similar to diff's. Although the two are not interoperable, they operate in the same manner: by determination of where the files differ and description of the differences. Therewith, a "new" file can be reconstructed from an "old" file plus a patch -- hypothetically, with according decrease of network bandwidth load. Although the script is a few hundreds of lines long, the scanner (the part that goes through the file looking for the interesting bits: such as, in this case, the positions where the new file differs from the old) is one tenth that size. As you've observed in my other software, I do without proper parsers & grammar. This renders my work brief, vulgar, and full of bugs, but sometimes legible. def diff (old_lines, new_lines): #fmt: old_offset old_lines new_lines\nlines\n patch_file = [ patch_copacetic_leadin ]; scan_line = ""; # Compute MD5 checksums for both files... old_md5sum = hashlib.md5(); for line in old_lines: old_md5sum.update(line); old_md5sum = old_md5sum.hexdigest(); scan_line = "%s\t" % (old_md5sum); new_md5sum = hashlib.md5(); for line in new_lines: new_md5sum.update(line); new_md5sum = new_md5sum.hexdigest(); if new_md5sum == old_md5sum: return None; # same file? then no patch req'd. scan_line += "%s\n" % (new_md5sum); patch_file.append(scan_line); # Second line: old_md5 new_md5 oi = 0; ol = len(old_lines); ni = 0; nl = len(new_lines); tally = 0; scan_line; unique_new_lines = set(new_lines) - set(old_lines); while ni < nl: # 2 phases: scan "same" lines, then diff lines oi = 0; tally = 0; while oi < ol and old_lines[oi] != new_lines[ni]: oi += 1; scan_line = "%d\t" % (oi); #Index in "old" file to cat some of its lines while oi < ol and ni < nl and old_lines[oi] == new_lines[ni]: tally += 1; ni += 1; oi += 1; scan_line += "%d\t" % (tally); # Number of lines to cat from "old" file tally = 0; next_ni = ni; while ni < nl and new_lines[next_ni] in unique_new_lines: tally += 1; next_ni += 1; scan_line += "%d\n" % (tally); # Number of lines to cat from "new" file patch_file.append(scan_line); patch_file.extend(new_lines[ni : next_ni]); ni = next_ni; # end while (scan the files, outputting the patch protocol format) return patch_file; # end function diff: returns diff-style patches as writelines() compatible lists Concise and transpicuous: 1. Tally runs of lines that already existed in the old file. (Scan phase.) 2. Tally runs of lines that do not exist in the old file. (Diff phase.) 3. Print a patch format that permits ordered reconstitution of the lines. 4. Repeat until the entire new file can be reconstructed from patch + old. Here, Python's set()s abstract away a tedious series of repetitive scans. Without set or a like data type, I'd have to either hash the "old" file's lines myself (and waste time writing another binary tree) or loop through it all again and again for each line of the new file. (That would be due to the fact that, if lines had been moved about instead of simply moved apart by interjection, then a lockstep scanner would mistakenly skip some and the patch file would be larger.) There is no capacity to patch binary files, but DW still detects when they have changed, and will write a copy into the patch directory. I assume that changes to binary files are due to transcoding, and therefore the patch'd be just as big -- some kinds of binary files, such as SQL databases, don't behave this way and can be patched in the same manner as I patch text files, but I don't use them. (If you extend the algorithm to databases or executables, don't forget to review the pertinent file formats and open the files in binary mode. :) The rest of the script is a wrapper handling directory traversal and file I/O. As `info diff` artfully states, "computer users often find occasion to ask how 2 files differ." The utility of a script like DiffWalk is therefore not limited to patching, but compression protocol is its primary employment on my system. (I still use `diff` for quotidian difference queries because DW isn't in my $PATH.) Likewise, the automatic collation of updates, such as moved and deleted files, is a pleasant amelioration to the task of finding what's changed in an archive since the last published edition. DiffWalk now handles these tasks for me. If you'd like a better solution to the "Briefcase Problem" (how to synchronize files across multiple installations with minimal time and fuss), don't forget to stop by the manual pages for "diff", "patch", and "rsync".
(I have implemented the Trivial File Transfer Protocol, revision 2, in this milestone snapshot. If you have dealt with reprogramming your home router, you may have encountered TFTP. Although other clients presently exist on Linux and elsewhere, I have implemented the protocol with a pair of Python scripts. You’ll need a Python interpreter, and possibly Administrator privileges (if the server requires them to open port 69), to run them. They can transfer files of size up to 32 Megabytes between any two computers communicating via UDP/IP. Warning: you may need to pull out your metaphorical monkey wrench and tweak the network timeout, or other parameters, in both the client and server before they work to your specification. You can also use TFTP to copy files on your local machine, if for whatever reason you need some replacement for the cp command. Links, courtesy of MediaFire, follow:
Executable source code (the programs themselves, ready to run on your computer): http://www.mediafire.com/file/rh5fmfq8xcmb54r/mlptk-2017-01-07.zip
Candy-colored source code (the pretty colors help me read, maybe they’ll help you too?): http://www.mediafire.com/file/llfacv6t61z67iz/mlptk-src-hilite-2017-01-07.zip
My life in a book (this is what YOUR book can look like, if you learn to use my automatic typesetter and tweak it to make it your own!): http://www.mediafire.com/file/ju972na22uljbtw/mlptk-book-2017-01-07.zip
Title is a tediously long pun on "Pan-Seared Programming" from the last lecture. Key: mechanism to operate an electric circuit, as in a keyboard. Emporium: ein handelsplatz; or, perhaps, the brain. Empyreuma: the smell/taste of organic matter burnt in a close vessel (as, pans). Lignite: intermediate between peat & bituminous coal. Empyreumatic odor. Pignite: Pokémon from Black/White. Related to Emboar & Tepig (ember & tepid). Pygmalion (Greek myth): a king; sculptor of Galatea, who Aphrodite animated. A few more ideas that pop up often in the study of computer programming: which, by the way, is not computer science. (Science isn't as much artifice as record- keeping, and the records themselves are the artifact.) MODULARITY As Eric Steven Raymond of Thyrsus Enterprises writes in "The Art of Unix Programming," "keep it simple, stupid." If you can take your programs apart, and then put them back together like Lego(TM) blocks, you can craft reusable parts. CLASSES A kind of object with methods (functions) attached. These are an idiom that lets you lump together all your program's logic with all of its data: then you can take the class out of the program it's in, to put it in another one. _However,_ I have been writing occasionally for nearly twenty years (since I was thirteen) and here's my advice: don't bother with classes unless you're preparing somewhat for a team effort (in which case you're a "class" actor: the other programmers are working on other classes, or methods you aren't), think your code would gain from the encapsulation (perhaps you find it easier to read?), or figure there's a burning need for a standardized interface to whatever you've written (unlikely because you've probably written something to suit one of your immediate needs: standards rarely evolve on their own from individual effort; they're written to the specifications of consortia because one alone doesn't see what others need). Just write your code however works, and save the labels and diagrams for some time when you have time to doodle pictures in the margins of your notebook, or when you _absolutely cannot_ comprehend the whole at once. UNIONS This is a kind of data structure in C. I bet you're thinking "oh, those fuddy- duddy old C dinosaurs, they don't know what progress is really about!" Ah, but you'll see this ancient relic time and again. Even if your language doesn't let you handle the bytes themselves, you've got some sort of interface to them, and even if you don't need to convert between an integer and four ASCII characters with zero processing time, you'll still need to convert various data of course. Classes then arise which simulate the behavior of unions, storing the same datum in multiple different formats or converting back and forth between them. (Cue the scene from _Jurassic Park,_ the film based on Michael Crichton's book, where the velociraptor peeks its head through the curtains at a half-scaffolded tourist resort. Those damn dinosaurs just don't know when to quit!) ACTUALLY, VOID POINTERS WERE WHAT I WAS THINKING OF HERE The most amusing use of void*s I've imagined is to implement the type definition for parser tokens in a LALR parser. Suppose the parser is from a BNF grammar: then the productions are functions receiving tokens as arguments and returning a token. Of course nothing's stopping you from knowing their return types already, but what if you want to (slow the algorithm down) add a layer of indirection to wrap the subroutines, perhaps by routing everything via a vector table, and now for whatever reason you actually _can't_ know the return types ahead of time? Then of course you cast the return value of the function as whatever type fits. ATOMICITY, OPERATOR OVERLOADING, TYPEDEF, AND WRAPPERS Washing brights vs darks, convenience, convenience, & convenience, respectively. Don't forget: convenience helps you later, _when_ you review your code. LINKED LISTS These are a treelike structure, or should I say a grasslike structure. I covered binary trees at some length in my fourth post, titled "On Loggin'." RECURSION The reason why you need recursion is to execute depth-first searches, basically. You want to get partway through the breadth of whatever you're doing at this level of recursion, then set that stuff aside until you've dealt with something immensely more important that you encountered partway through the breadth. Don't confuse this with realtime operating systems (different than realtime priority) or with interrupt handling, because depth-first searching is far different than those other three topics (which each deserve lectures I don't plan to write). REALTIME OPERATING SYSTEMS, REALTIME PRIORITY, INTERRUPT HANDLING Jet airplanes, video games versus file indexing, & how not to save your sanity. GENERATORS A paradigm appearing in such pleasant languages as Python and Icon. Generators are functions that yield, instead of return: they act "pause-able," and that is plausible because sometimes you really don't want to copy-and-paste a block of code to compute intermediate values without losing execution context. Generators are the breadth-first search to recursion's depth-first search, but of course search algorithms aren't all these idioms are good for. Suppose you wanted to iterate an N-ary counter over its permutations. (This is similar to how you configure anagrams of a word, although those are combinations -- for which, see itertools.combinations in the Python documentation, or any of the texts on discrete mathematics that deal with combinatorics.) Now, an N-ary counter looks a lot like this, but you probably don't want a bunch of these... var items = new Array(A, B, C, D, ...); // ... tedious ... var L = items.length; // ... lines ... var nary = new Array(L); // ... of code ... for (var i = 0; i < L; nary[i++] = 0) ; // ... cluttering ... for (var i = L - 1; i >= 0 && ++nary[i] == L; // ... all ... nary[i--] = ((i < 0) ? undefined : 0) // ... your other ... ) ; // end for (incrementation) // ... computations ... ... in the middle of some other code that's doing somewhat tangentially related. So, you write a generator: it takes the N-ary counter by reference, then runs an incrementation loop to update it as desired. The counter is incremented, where- upon control returns to whatever you were doing in the first place. Voila! (This might not seem important, but it is when your screen size is 80 by 24.) NOODLES AND DOODLES, POMS ON YOUR POODLES, OODLES AND OODLES OF KITS & CABOODLES (Boodle (v.t.): swindle, con, deceive. Boodle (n.): gimmick, device, strategy.) Because this lecture consumed only about a half of the available ten thousand characters permissible in a WordPress article, here's a PowerPoint-like summary that I was doodling in the margins because I couldn't concentrate on real work. Modularity: perhaps w/ especial ref to The Art of Unix Programming. "K.I.S.S." Why modularity is important: take programs apart, put them together like legos. Data structures: unions, classes. Why structures are important: atomicity, op overloading, typedefs, wrappers. linked lists: single, double, circular. Trees. Binary trees covered in wp04?? recursion: tree traversal, data aggregation, regular expressions -- "bookmarks" Generators. Perhaps illustrate by reference to an N-ary counter? AFTER-CLASS DISCUSSION WITH ONE HELL OF A GROUCHY ETHICS PROFESSOR Suppose someone is in a coma and their standing directive requests you to play some music for them at a certain time of day. How can you be sure the music is not what is keeping them in a coma, or that they even like it at all? Having experienced death firsthand, when I cut myself & bled with comical inefficiency, I can tell you that only the dying was worth it. The pain was not, and I assure you that my entire sensorium was painful for a while there -- even though I had only a few small lacerations. Death was less unpleasant with less sensory input. I even got sick of the lightbulb -- imagine that! I dragged myself out of the lukewarm bathtub to switch the thing off, and then realized that I was probably not going to die of exsanguination any time soon and went for a snack instead. AFTER-CLASS DISCUSSION WITH ONE HELL OF A GROUCH "You need help! You are insane!" My 1,000 pages of analytical logic versus your plaintive bleat.
(Here be an update, as of November 8th, 2016. Me old war wound be actin’ up too much, and I think these’ll be the last for some time.
Ahoy, mateys. Today be the nineteenth of September — ye’d be better knowin’ it as International Talk Like A Pirate Day — and I’ll wager that upon this fine occasion ye’d be askin’ yerselves: “where’s me booty? ”
Well, and I’d make a poor excuse for a captain if I couldn’t deliver ye at least that! (But avast: ye might be findin’ it somewhat unholy, and parental discretion be even more advisable than in previous revisions.) I have prepared for ye a fine trove o’ source code, the likes of which are fit for Kings. Although me mother be the only one likely to find it interestin’, I’ve also put the finishin’ touches on me preliminary sketch of a typesetter for me book: “Yawnie’s Whole: the Complete Yawnie, for the Yawnie Enthusiast.” These be available in three chests, or what ye might be callin’ “Zip Arr-chives,” which I be uploadin’ to Mediafire as per usual.
Me latest revision of MLPTK be here…
… and be comprisin’ not much different from the last MLPTK, again as usual, except that I were fixin’ bugs. I report with most contrition that Polyfac be a failure: I be tryin’ to return me attention to the other tasks I failed to complete this year.
If ye prefer to be tastin’ th’ rainbow, a set of syntax-highlighted HTML documents illustratin’ the source code be here…
… they scry as nearly as possible alike to me own development environment.
Would ye like me book? I be certain to update and revise it as time be passin’, but who knows if me accounts shan’t be commandeered in the interstice? If ye be at all interested, don’t hesitate: supplies be unlimited, but tempus fugit…
… and, someday, me literature be gone forever, as literature inevitably shall.
And there be little more to say about this revision, as I’ve prepared no new lectures since April.
In the meantime, have ye noticed how beautiful life can be sometimes? Quite apart from th’ hardship and pain, there be especial bounty of resources. If ye be readin’ this, then ye would be privileged to Internet access, which are a rare treasure: there be all sorts o’ literature & art to be found, plenty of amusin’ diversions, and certainly no shortage of comely wenches to descry.
Me meaning be: ye could probably spend yer whole lives havin’ not a thing but a netbook computer, occasional access to electrical power, and some sort o’ shelter to protect ye from the elements. A “sex tent,” if ye will: just be addin’ some wenches. Why, I can imagine that no few individuals upon this blasted globe could be livin’ their lives contented with a shelter and a wench — wenches of the world bein’ blessed not to be needin’ anywench else.
Childhood be another of those times. As I grew, I were witness to what some would be describin’ as the “Wild West” of the World Wide Web. Nearly every outlet of popular culture were findin’ its way into troves and hoardes shared worldwide by generous scoundrels (and belligerent litigious bilge rats) to an audience of hundreds of millions. The vast serpent of DHTML and jQuery had only just been sighted far afore, and the stars fated to portend swashbucklin’ adventure at every second of the compass.
There was, too, a massive population of reputable sailors upon the vast waters of cyberspace. I remember some of the finest: OverClocked ReMix, VGMusic. Angelfire, Tripod, and Geocities. Neopets. The Merchant Guild. 4chan. So many more motes be floatin’ in the eye of history that I cannot even recount. Ah, the world were bigger then, and me eyes wide in childlike wonder.
Well, and it were the best of times, but me swashbucklin’ days be sadly behind me. (Arr, insofar as I cannot swash without me bucks! Besides that, me galleon be in disrepair, and overhaul be veritably a tribulation. However, as usual, be sendin’ me no money, for I cannot guarantee that it shall ever arrive; nor could I be certain it would help if it did.) As it happened, although I were studyin’ me life’s work throughout me life, me attention were turnin’ too late to serious programmin’ (peradventure, alas!), and circumstances be such that I envision failure to accomplish writing the parts of me portfolio I’d intended to finish this year.
(Happily I were not askin’ for research grants, considerin’ me doldrums.)
I be in pain; and, in light of this, tried to pass along what few ideas I were able to sustain the concentration to write before I be entirely unable to do so. They be in me ephemerides, toward page 950.
The spring be another of those times when life be less painful than it’s usually. I tell ye there be nothing like the sensation of warm sunlight on yer skin for the first time in months. Which are even assumin’ ye survived th’ winter — in the frigid North, for example, ye might be a popsicle if ye aren’t careful.
And let’s be not forgettin’ lemons…
Ah, but me ramblin’ be more piteous than a scurvy dog.
Enjoy me work.
Here be a ninja update fer th’ new year, 2017.
Ever wanted t’ shred data? Here be a tip:
dd -if /dev/random -of /dev/sda
will shred your ENTIRE HARD DISK /dev/sda irreversibly.
The file system be destroyed the instant you hit enter. There be no confirmation.
Shred it all night long, then when ye wake in the mornin’ do this before work:
dd -if /dev/urandom -of /dev/sda
to drop a load on yer disk that be heavier’n fifteen spars on a dead man’s chest.
Seriously. This be how to erase yer disks so thoroughly even the C.I.A. shall never espy yer dirty secrets.
Sleep tight, mateys.
Here are some updates on my progress:
^- This one contains a snapshot of my MLPTK directory as of last night shortly before I endured my nightly battle with pain that keeps me awake. It’s a couple’a megs (zipped) or about three megabytes (inflated). Download this if you only want the MLPTK software and reference materials.
^- This one contains a tasty treat! I have rewritten my automatic typesetter for today’s report, and there are now available some syntax highlighted HTML documents that make the code very much easier to read. There are some cute PDFs, too, for printing. DocBook was a very convenient format to get started with, but I hope with future revisions I will graduate from the training wheels and learn to use LaTeX / PostScript. The report is about thirty megabytes (zipped) or fifty megabytes (inflated). Download this if you would like to preserve my work in “Dead Tree” format. (Yes, I’m pretty sure that all the PDFs with my name on them are actually my own work, thus the “Author” bit in the typesetter. I deleted all the PDFs that I identified as not being my own work. Yes, it will require over four hundred pages if you print it. No, I’m not done writing.)
I have been working on MLPTK, and other projects, but have nothing of any value to report. Sadly, I stalled-out on QL and everything, and I don’t think I can publish anything complete in 2016 as I had hoped.
However, I _have_ made a good run at finishing every part of QL that I set out to write last Christmas. Also, I have run a quick test on the Firefox bundled with a recent version of Lubuntu: ironically, the _horrendous QL speed bug_ has no perceptible effect on execution time in that version! (I’m still going to try and fix it, though, because my development machine is still on the elder version.)
Since the last revision I have also written a few other modules, and fixed sundry bugs. See the history log for more details.
P.S. The header image easter-egg has also changed. Again.