I heard you like programming, right…
Lots of hours have been put into the clipper by now. As you may know, Kindle devices store this data on plain .TXT files, put together chronologically. If it’s a pain to navigate these text files on Desktop, it’s simply excruciating to try to do so on the Kindle itself (and by the way, if you are interested in readers I’ve heard that the Kobo Glo HD have a proper clipping system working with a hidden SQLite database). Having soooo many notes in my Kindle the thing had to be fixed and, as it usually happens, I wasn’t the first one to try to achieve this; several paid and free, proprietary and open attempts were scattered out there. It didn’t take much time to adapt some open source, ad-hoc code that I found, written to perform exactly what I wanted. But then I discovered that there were loads of different formats, and I started building on that, adapting my code to be as flexible as possible. Then I started thinking about some cool functionalities, and thus I sat down and wrote. After that, well, it was almost defiant not to invest some time in a GUI… and so on. Sadly I didn’t know of the existence of Wakatime when I started it all, innocently trying to find a better way to manage the notes and highlights that I frequently take on my Kindle reads, but wouldn’t be surprised to see it go above the two or three hundred.
But hey, it is done! Even if I started the project aiming to solve said problem, I kept getting deeper and taking the advantage to learn new tools, skills, languages, and environments. I think am way better at debugging after following the man in black across the desert, as an industrious (if not particularly efficient, at least at the beginning) gunslinger. Needless to say, this has been my first parser. Heck, it’s been my first big piece of non-games software! And it has also been my first incursion into WPF, an overall highly challenging, rewarding, frustrating and ultimately enriching experience. As I gather my lessons and evaluate the progress I’m hoping this could be of help for anyone out there mimicking the effort; even if I’m not a coding giant I’m pretty sure that many will benefit of standing on my shoulders. As importantly, if you’re curious about programming but haven’t put much effort on it, I’d like to offer you this information. It is not a tutorial nor a postmortem, but the views of another beginner learning some amazing stuff and getting a better grasp on what he does. Let’s go for it, and happy coding!
General lessons (wot I learned and/or did work for me)
- Everything is bigger and more complicated than you think (and probably than it looks): I thought I got this clear while working on games, but as usual you need quite a bunch of cups of it to really get it. Normally there are no such things as ‘I just will that controller modifying this property’ or ‘I bet this functionality can be covered with a small struct’, or really any other sentence that starts with a ‘probably it’s just’. Spoiler: most likely it isn’t. Unless you’re quite experienced in the problem AND the tools, expect things getting messy, and commit yourself to refactor and reshape your code from the very early stages, rather than pushing forward onto increasingly weird but coincidentally functional code. Those ‘minor tweaks’ and modifications will haunt you later, and it always has the potential to get worse. I’ll definitely bear that in mind from now on.
- Design your program before start coding: As in game design, it never ceases to amaze me the power that a simple notebook has. Of course, you can go full UML, but in such a project like the clipper a notebook will suffice. Draw the system, dig into its relations, challenge your assumptions. Do it even more intensively if you’re working under Object-Oriented Programming principles. I recall a beautiful concept that I studied back in my science degree, about approaching problems in terms of a black box, inputs, and outputs. In the magnificent coding book “The Pragmatic Programmer” there’s quite a compatible concept: Design by contract, a “technique that focuses on documenting (and agreeing to) the rights and responsibilities of software modules to ensure program correctness” that approach problems in terms of preconditions, postconditions and class invariants. Think about that before you actually sit coding, avoid programming by coincidence, and bear in mind that every proper OOP class is shy (exposes only the necessary details, see encapsulation.
- Search efficiently, not extensively: This is again a matter of experience, but after a few hundred hours working on the Clipper, I realise that I have become way better isolating and abstracting problems (or, in other words, I now see how many pointless articles, code snippets or legacy solutions I have been through). I’d say it’s good to take every read with a grain of salt, and it worths to put some effort on a problem’s dissection. It takes patience, but in this context sniping is better than machine gunning. It won’t only save you time, but it will teach you the general challenges behind a problem, so that you can reuse that knowledge on other languages or tools.
- Always, always always always use source control: I didn’t start using Git until I was halfway through the project, and that was a very, very big mistake. Even if I got lucky and I didn’t suffer from any data loss or drive problem, there’s hours and hours of work in which the code evolved in very curious ways, and there’s not a backlog covering it. Yeah you might be working on your own, yeah I know you’re pretty sure about just being toying around with this or that new tool. Perfect: start using source control anyway, no matter what. And remember to commit often. As for me, I’ve found Sourcetree to be of great help while managing my repositories. If you’re not the command line master that you’d like and need a friendly GUI to keep your projects clean, consider giving it a go. Other options include GitHub and Gitkraken, programs that I used and definitely liked.
- Know your tools (both their strong points and limitations): Many times you will discover that while trying to get the damn thing working you were hammering a nail with the handle of a knife. Don’t worry, some very experienced and talented programmers confirm that it happens all the time. Move on and find the right tool (and toolset).
- Consistency is everything: That goes for naming conventions, comments, documentation and pretty much everything. It will make things easier for you, and for anyone having to understand your code for any reason (including helping you!).
- Look and feel is important: Or, how even if you completely rewrote the program users will most likely value the tiny, cosmetic improvements. Testing a bit and keeping an eye on user experience and feedback will definitely not hurt your project.
On parsers, WPF and .NET findings
- Learn concurrency, and make sure to understand threading even if it’s to work around it: I wish I had paid more attention to this. C# is a multi-paradigm programming language based on Object-Oriented and Imperative paradigms, this latter meaning that code is read strictly in sequence, moving line per line and item by item, “telling the computer what to do” in a sequence of statements. These bunch of statements can be organised in processes, each one managing this code in an isolated flow called a thread. But, of course, a program doesn’t just go in a sequence. What if there are several different tasks that should be executed simultaneously? What if some code shall be performed in the background? That’s what concurrency and multithreading programming are there for. How I discovered the thing: by having the user interface blocked every time I executed some resource-consuming task. The paramount of WPF beginner terrible mistakes, I learnt that the UI thread needs to be separated from other threads in order to keep up updating, thus remaining (and looking) responsive. Gladly .NET 4.5 introduced a simplified Asynchronous Programming with Async/Await, a nice addition that fixed almost all of my problems without having to go too far on Multithreading.
- If you’re into parsing, Regex is a must: Directly from C# documentation:
A regular expression is a pattern that the regular expression engine attempts to match in input text. A pattern consists of one or more character literals, operators, or constructs.
When you’re trying to convert a bunch of confusing text into neat objects with different fields, trimming the hell out of it and having the program understand what he is reading you need to use them efficiently. And it’s a very, very deep field of study in an on itself. But it’s worth it nonetheless.
- LINQ often has the solution for your syntax-related challenges: Contrary to the opinion of many, LINQ (Language Integrated Query) is not just used for database querying, even if it’s an advantage if you currently have a certain understanding of SQL. LINQ is a very powerful beast, and can be used to perform incredibly optimized operations on text with a very succinct chunks of code. Also, remember that of C# being a multi-paradigm language? LINQ is a good example of Functional Programming paradigm and capabilities.
- Time invested in getting acquainted with Anonymous Methods pay off: Say Delegates and Lambda expressions. The first ones are the base of Events systems (part of the lifeblood of WPF) and the second make many operations way easier (including LINQ ones), writing local functions that can be passed as arguments, simplifying syntax for delegates. Once you are comfortable with them they’ll become essential and really handy.
- Think of XAML as a totally different entity, because it is: When I started using WPF, I thought of XAML (Extensible Application Markup Language) as a nice addition. However, it is a whole world itself, allowing you to building interfaces, using a CSS-like system but automatically binding data, allowing for sort of magic behaviour. Many of the clippers functionalities may have been done through XAML as opposed to programmatically, and whatever your preferences it is necessary to be aware of it. Interfaces are hard, and all help is welcome!
- Model–view–viewmodel pattern would have probably made my life a bit easier: Very related to XAML, this pattern tries to isolate model (data) from view (interface) and connect the two of them through a viewmodel (intermediary view logic), separating concerns between the user interface controls and their logic (and effectively improving modularity, reusability and reducing coupling). More details here.
Phew, there! That’s it. Curious about the code or writing to contribute? Head to the links below.