| Astroblahhh Desktop v2.0 |
|
Wednesday, February 11th, 2015 14:22:48 GMT |
Software Release |
I finally managed to make a pretty much releasable edition of Astroblahhh Desktop v2.0.
The source code is hopeless mess which I can't even hope to perfect in a reasonable amount of time. I also didn't adequately update its documentation. But, it's amazing how well it works, despite such hideously messy source code.
I still use it very frequently because I still haven't completed Astroblahhh Desktop v3.0, which is in progress, but not useable yet.
More info is on Astroblahhh Desktop's home page.
Or, perhaps I'm only daydreaming. On the other hand, maybe the only thing that might make it impossible is the oppression caused by my and my family's financial struggles.
As usual, donations and microdonations and purchases of goods (still none available yet) and services (again, somewhat available, except for the fact that I keep avoiding even checking my email) are welcome.
Edit, 2:38 PM: Had to reupload the zip file several times, because I kept noticing little problems that had to be fixed. The timestamp of the hopefully final zip file is 2:31:20 PM EST, and it was uploaded probably less than 2-3 minutes after that.
Perhaps someday Astroblahhh Desktop might become a plausible rival to things like Evernote or Trello.
▲ Top ▼ Bottom △ TOC ↓ Down Up ↑
| Astroblahhh Desktop v2.3 |
|
Sunday, September 20th, 2015 20:36:25 GMT |
Software Release |
This is a relatively minor update of Astroblahhh Desktop compared to what I'm working on.
(Addition, 6:55 PM EDT: Reuploaded because I forgot to include the GNU Affero General Public License.)
And, I should point out that like all the previous releases, I still haven't gone to as much trouble as I should to try to make it easy for people other than myself to install and use. And it still contains a ton of legacy code I wrote years ago, which has many bad design issues you'd expect from a relatively inexperienced programmer, such as using global variables all over the place. (Hopefully most future versions will be better in these regards.)
But, the most notable thing I recall adding is the ability for the file organizer to calculate SHA-512 checksums (also known as SHA-512 hashes) of files, and store them in file databases. Plus other things related to SHA-512 checksums, to compare checksums in two different databases, search for checksums, etc.
I did this simply so I could be more certain that my backup files in various disks (or discs) are intact. (Also, it's nice to be able to easily locate and group together identical files.)
But, that article made me realize (or reminded me) that I had hardly any idea how much bitrot my files might already be suffering. So, several months ago, I decided to temporarily set aside my work on Astroblahhh Desktop 3.0 (and other things), and solve this problem first. I think I finished this stuff back in June or July.
That article mentions checksumming a lot, so, that gave me the idea to add the new checksum features to Astroblahhh Desktop. Which aren't the same as the checksum features mentioned in that article - instead of calculating checksums for "blocks", Astroblahhh Desktop calculates checksums for the entire files.
Prior to this, I never did much of anything with checksums until I made the downloader script portion of my Puppy Linux Setup Kit, since I needed a quick and easy way to verify that downloads had been successful.
I wish I had paid attention to the concept of checksums much sooner, since it seems very clever and useful (even though of course I don't fully understand it).
A quite slow process, since I have a lot of files - and after a major reorganization of my disks a while back, I hadn't resynced my databases with the new actual locations of all my files yet, so that slowed me down too. (For the resyncing, I had to use Astroblahhh Desktop's "mass locate" and "mirrorer" features a lot, which made it quite a lot easier, but it was still not as convenient and automated as would be ideal.)
Another thing that made calculating checksums slower was, I made a way to make SHA-512 checksums for what I call "paired" folders - which are folders which often get created to hold the images, CSS files, etc. of web page files you save with Firefox and other browsers.
(I save a lot of web page files, so, even from the beginning of the file organizer portion of the Astroblahhh Desktop project, I went out a lot out of my way to build in capabilities specifically to deal with web files and their paired folders, so it wouldn't be as easy to accidentally separate web files from their paired folders.)
This is quite possibly stupid in some way I haven't yet thought of. (If so, I'd love to hear about it, or any other feedback. Comments are welcome at Eryss.Com Forum or can be sent privately to me.)
But, it seems to achieve the goal of reliably creating the same checksum for different copies of the same paired folder - so, I'm happy enough with it for now, even though calculating the checksums of paired folders is often much slower than calculating checksums for lone files.
One of my new scripts enables me to compare the SHA-512 checksums in two different databases, so, after the slow chore of adding checksums to my databases, I was able to quickly and easily verify that my various redundant backups have all the same checksums in them. Happily, I haven't detected any bitrot so far.
It's so nice to be able to know with far more certainty that my files are intact, rather than just making lots of redundant backups and mostly just hoping they're all truly identical. (I did sometimes use the "diff" command on them, though.)
Most future versions will hopefully have the ability to run as either a PHP-GTK true desktop app or a web app, and maybe even also a console/command line app.
Also, since I was able to get PHP-GTK to work with PHP 5.6.13, there's nothing holding me back anymore from starting to use whatever fancy new PHP features I was missing out on as a result of sticking with PHP 5.2.17 for so long.
I hope in the end, Astroblahhh Desktop will be easily installable, useable and useful even in Windows and MacOS, as well as Linux. (Well, it already is mostly tailored to Windows, since it was first written while I was still mostly a Windows user, and, up until now, I usually used it in a Windows XP VirtualBox, even though in 2011 I added some compatibility with Linux.)
They're still useful, even if they're not as mindblowingly flexible as OrientDB is, and even if I'm itching to build an OrientDB-based possible successor to Astroblahhh Desktop.
Astroblahhh Desktop's existing databases are also a lot less mind-boggling to me than my graph database-related daydreams.
But probably what's making the graph database stuff a lot more mind-boggling than graph databases normally are is my obsessive struggling with trying to design my ideal database software, with temporal-database fanciness, rewindable history, provenance of data instead of just data alone, multiple users, etc., etc. I haven't even started coding it yet, I've just been jotting down tons of notes and ideas in VUE concept maps and NoteCases.
All my lofty ideas seem a lot more feasible with a graph database rather than MySQL/MariaDB or SQLite. But, those ideas might be so difficult I'll have to give up in the end, and do something easier.
Fortunately, even something easier will probably still be quite powerful and useful, and quite possibly better than Astroblahhh Desktop with less effort.
Anyway, I've only struggled with designing my ideal graph database since late August, and only quite recently got PhpOrient to work (because I had to go to the trouble to update Astroblahhh GLMP-GTK to 2.0), so I'm not tired of the struggle yet.
I'm kind of fed up with the Thunderbird email client's overly bloated profiles and how complicated it is to deal with my email on multiple computers, since I usually prefer downloading my mail with POP3 rather than leaving it on the server using IMAP, and I don't want to have to mess around with figuring out how to merge different Thunderbird profiles from different computers together.
And I also want to escape Gmail, since it's closed source and a Service-as-a-Software-Substitute - SaaSS.
I somehow managed to compile PHP's IMAP extension (which, among other things, also can handle POP3 and even NNTP, the Usenet protocol). And then managed to figure out how to access one of my DreamHost email accounts with a PHP console script on my own computer (rather than uploaded to DreamHost).
So, my daydream of writing my own email software is starting to seem truly feasible. It may seem silly to reinvent such a wheel, rather than just look harder for existing email software that I like more than Thunderbird - but for some reason I feel inspired to go to the trouble.
That also makes possible an up-to-date PHP 5.6.13+PHP-GTK edition of the astrology software Eryss - though I'm probably not very likely to work on that in the very near future, because astrology software is nowhere near as useful on a practical level as my other projects.
But, I haven't yet written the build-from-scratch scripts. I'll most likely get around to it when I get around to going back to Lighthouse 64 Puppy Linux 6.02 Beta 2, since I haven't built Astroblahhh GLMP-GTK 2.0 for LH64 yet.
Despite all the organizer software I write, I'm still a lot more disorganized than I should be, so, I didn't keep very careful track of what I added to Astroblahhh Desktop since the previous release.
This was inspired by a scary article I read which mentioned the concept of bitrot. Here is that article, from Ars Technica:
"We look at the amazing features in ZFS and btrfs - and why you need them"
I was quite dismayed by that article, but didn't feel up to figuring out how to implement bitrot countermeasures such as switching to the ZFS or btrfs filesystem, or setting up some kind of automated RAID backups involving multiple hard disks.
After (and even while) building the SHA-512 checksum features into Astroblahhh Desktop, I went through my old file archives and made SHA-512 checksums for most everything.
If I recall correctly what my paired folder SHA-512 checksum-making code does - all the files contained in the paired folder are checksummed, with each of their checksums added to a single long string, then a single SHA-512 checksum is calculated for that long string of checksums, and that gets used as the paired folder's checksum.
So, now, if bitrot ever occurs, at least I'll hopefully be able to detect it at some point in the future. And, amongst all my many redundant backup files, I'll be able to tell which backups are intact and which are corrupted.
This might turn out to be one of the last, uh, conventional versions of Astroblahhh Desktop. (Though a web app that isn't intended to run on a web server on the internet isn't all that common/conventional. But, that's what Astroblahhh Desktop always was. So, this version is at least typical/conventional in the context of Astroblahhh Desktop's own history.)
Despite my recently-discovered adoration of graph databases, I still want to preserve Astroblahhh Desktop's ability to use MySQL/MariaDB and SQLite databases. (SQLite is currently useable in a not-yet-released version of Astroblahhh Desktop.)
Donations are microdonations are welcome. Meanwhile, goods and services still aren't really available yet. But services might become more available if/when I finally solve the various nuisances with my email.
So, I'm seriously thinking about adding an email organizer to Astroblahhh Desktop and writing my own email-downloading software.
I also managed to compile and use PHP-SWEPH 1.80 with PHP 5.6.13 in Astroblahhh GLMP-GTK 2.0. Just thought I'd point out that it's possible.
I haven't released any compiled Astroblahhh GLMP-GTK 2.0 binaries, and maybe won't at all, because I think it might be better to release Astroblahhh GLMP-GTK 2.0 as build-from-scratch scripts in another update of my Puppy Linux Setup Kit.
Again, donations and microdonations are welcome.
▲ Top ▼ Bottom △ TOC ↓ Down Up ↑
| Warning about character sets and Astroblahhh Desktop; other news on ABDesktop 2.4 & Astroblahhh GLMP-GTK 2.0 |
|
Wednesday, September 30th, 2015 03:48:10 GMT |
I just added this new warning to Astroblahhh Desktop's home page, just in case there actually might be any users of Astroblahhh Desktop in the world besides me:
I just recently found out editing a database record while using a different character set than that record was created with can mess up unusual text characters. (I stumbled across this because my Windows XP installation of Astroblahhh Desktop - with web browsers I haven't updated for many years - and my Linux installation of Astroblahhh Desktop - which uses Firefox 32 - happen to use different default character sets.)
So, it looks like I need to upgrade Astroblahhh Desktop to store all database text in Unicode (UTF-8) format. But, if you always use Astroblahhh Desktop on the same computer platform with the same default character set, hopefully that's safe.
Also, not even the latest-released versions are fully compatible with Linux yet, despite some efforts I made in 2011. But, I'm definitely working on that too. I love VirtualBox, but I don't love having to use it (or Windows XP) to use Astroblahhh Desktop.
Luckily, I didn't lose any data because I was still only upgrading and testing Astroblahhh Desktop 2.4, rather than using it on my real databases.
But, even though I might be the only serious user of Astroblahhh Desktop in the world, I thought I should point out that warning just in case.
Even the version of MariaDB I used in Astroblahhh GLMP-GTK 1.0 - MariaDB 5.3.12 - doesn't have "utf8mb4" encoding. Unless I'm mistaken, judging by this page, only MariaDB 5.5 and up have that.
So, though I was originally actually tempted to just stay with MariaDB 5.3.12 in the not-yet-released Astroblahhh GLMP-GTK 2.0, it looks like I need to upgrade.
And on Sept. 30, 2015 - my web host, DreamHost, is going to automatically update my sites' databases to MySQL 5.6.
So, to maintain compatibility with MySQL and the web host I use, Astroblahhh GLMP-GTK 2.0 is probably going to use MySQL 5.6.26 rather than MariaDB, even though I still like MariaDB.
I even was able to read my old Astroblahhh Desktop databases without changing them at all for use in MySQL 5.6.26. And, this all still works using the old version of the Hiawatha web server included with Lucid Puppy Linux 5.2.8 version 004.
I think it's neat that things still work so well despite not upgrading Hiawatha - though I probably should add an updated Hiawatha to Astroblahhh GLMP-GTK 2.0 nonetheless.
I'm particularly interested in clues on what to do about MySQL's libaio dependency. This hasn't been too easy for me to fix as a Puppy Linux user, because I don't have a package manager like "apt" or "yum", and practically the only "solution" I can find on the web is advice to use your package manager to download libaio.
But even if I had a package manager, I'd still rather find libaio's source code and compile it myself from scratch, because I don't like the idea of being dependent on a package manager, or even on an internet connection.
I'm also interested in clues on whether or not doing without libaio is likely to lead to catastrophe.Warning, Sept. 29/30, 2015: Please be wary of using the same Astroblahhh Desktop databases on two very different computer platforms, such as Windows XP and Linux, or even the same platform but with different web browsers, such as a newer browser which is likely to use Unicode as its default character set, and an older browser which doesn't default to Unicode.
I've been working on Astroblahhh Desktop v2.4 for several days now. My goal with 2.4 was simply to update as little as possible to get Astroblahhh Desktop working properly in Linux (and PHP 5.6.13). The update was seeming pretty easy overall, until I ran into that Unicode issue.
One surprising problem I ran into was that, according to this page, MySQL databases had rather poor Unicode UTF-8 support until MySQL 5.5.3, which was released in early 2010. (Which is very old, but still newer than the ancient edition of XAMPP I use in my Windows XP VirtualBox for Astroblahhh Desktop.) And, for decent Unicode UTF-8 support in MySQL, you have to use "utf8mb4" encoding rather than just "utf8".
I managed to figure out how to compile and use Cmake, so I was able to build the latest MariaDB - but then I found out from this page that starting with version 10.0, MariaDB has started to diverge more from MySQL than MariaDB did in the past.
I couldn't figure out how to install something MySQL supposedly needed called libaio, but I managed to compile MySQL anyway, and it seems so far to work, judging by the fact that I was able to look at MySQL 5.6.26 databases in PhpMyAdmin 4.5.0.2.
Comments, suggestions, etc. are welcome at the Eryss.Com Forum, either on this thread or elsewhere.
▲ Top ▼ Bottom △ TOC ↓ Down Up ↑
| Astroblahhh Desktop v2.4 |
|
Wednesday, October 14th, 2015 20:15:30 GMT |
Software Release |
Astroblahhh Desktop 2.4 is another "conventional" release of Astroblahhh Desktop. (Conventional in the context of its own unconventional history.)
That means it's still a horrible mess (though remarkably well-functioning), and it's only runnable as a web app - but since it probably has major security problems, you shouldn't run it while connected to the internet. And it's still nowhere near as polished as I'd like it to be for people other than myself to use.
I haven't tested it extensively in Astroblahhh GLMP-GTK 1.0, and I also haven't even tested it at all in XAMPP in Windows XP.
My goal with this version was simply to make it possible (again) for me to use ABDesktop natively in Linux, so I wouldn't have to run ABDesktop inside a Windows-XP-running VirtualBox with no internet connection.
I haven't fully tested Astroblahhh Desktop, but, except for various known problems (and probably some unknown problems), it seems to largely work in both PHP 5.2.17 (using Astroblahhh GLMP-GTK 1.0), and PHP 5.6.13 (using my not-yet-released Astroblahhh GLMP-GTK 2.0). I haven't tested this version at all in XAMPP in a Windows XP VirtualBox.
Also, I haven't tested the file-moving features at all lately, so I recommend avoiding them.
A pretty annoying problem with Astroblahhh GLMP-GTK 1.0 is that it can't create JPG thumbnails, because when I compiled PHP 5.2.17, I didn't realize it was necessary to manually add JPG capabilities to the ./configure command line.
An even worse problem is, I still didn't get around to converting everything to use Unicode UTF-8 encoding. So, the warning I posted on Sept. 29th/30th still applies.
I also hastily added a universal read-only mode, and a non-universal read-only mode - either of which you can turn on by editing the file "lib-abdesk.php" in the "abdesk" folder. The non-universal read-only mode makes all databases read-only by default except the ones that are added to the array $list_of_writable_dbs, stored inside "lists_of_all_your_dbs.php" inside the "custom" folder.
Both read-only modes also block the ConnectAsRoot() MySQL database connection function from connecting at all, even to databases listed as writable.
The read-only restriction is enforced by using a MySQL user which is only allowed to use the SELECT command. That user can be created using the new "Create Read-Only User" script. I think the read-only features probably work, but, please be careful.
Rather than depending on the read-only features, I prefer to just copy my databases into my RAM disk. That keeps the originals safe, and makes the copies very fast to read.
And as it should be, in my opinion - because I agree with this article that case sensitivity in software tends to be evil. Nonetheless, I'll someday try to deal with the case sensitivity nuisance eventually - but, I think I'll make Astroblahhh Desktop optionally "enforce" case insensitivity, such as by refusing to move files with exactly the same name except different capitalization into the same directory.
To the file organizer, I added a rather half-assed ability to deal with symlinks. The file organizer recognizes symlinks, but doesn't store their destinations. It's easy for PHP to obtain a symlink's destination, but changing the database's schema to store this new info seemed like too much of a nuisance to undertake for an already nearly obsolete version of Astroblahhh Desktop.
However, again, I haven't tested the file-moving features at all lately, and I recommend avoiding them.
Also, Linux file permissions often prevent the file organizer from even being able to read files in many locations. That was such a nuisance (particularly when I couldn't figure out how to mount NTFS-formatted TrueCrypt volumes with appropriate permissions), I finally opted for the quite questionable workaround of slightly editing the Hiawatha web server's source code - as described in this Hiawatha Webserver forum post from Sept. 1, 2010, 11:13 - to make it possible to run Hiawatha as root.
One thing that makes this workaround a bit less terrible is the fact that I've returned to my old habit of only using Astroblahhh Desktop (or any web server software, especially the Hiawatha I modified to run as root) while disconnected from the internet.
In fact, I now just use a totally different computer for the internet - and, thanks to my Puppy Linux Setup Kit, doing that is actually pretty comfortable. (Except I have too little RAM, so I have to stay away from sites like Tumblr, and even Google Image Search can crash my entire system.) I don't even need a hard drive, and could even get by without a Flash drive if I didn't usually like to save things, or at least URLs.
Unfortunately, using a different than usual computer for the internet is part of the reason I've lately had even more difficulty than in the past with my email. I have my Thunderbird profiles on a hard drive in an external USB enclosure because they're so bloated they don't fit comfortably on my Flash drives, and I don't want to maybe wear out my Flash drives by downloading mail onto them. And, even though my external hard drive is technically portable, I still don't like moving it, and my internet computer is too far away to plug in my hard drive without moving it.
I guess probably using the file (record) adder and making image thumbnails is faster now too, but that was usually tolerably fast even with Astroblahhh Desktop inside VirtualBox.
I also am not eager to possibly wear out a USB Flash drive by storing and running a Windows XP VirtualBox on a Flash drive. Fortunately, larger and larger Flash drives are getting cheaper and cheaper, but, I still don't like to waste things or money if I can avoid it.
And I'm hoping even the not-yet-finished PHP-GTK version will also be able to run on pathetic computers with only 256 MB RAM. Or even less RAM would be nice.
However, having only 256 MB RAM or less might necessitate using a Puppy Linux full installation instead of a frugal RAM-disk-using installation, since Puppy plus Astroblahhh GLMP-GTK might use up too much RAM, even if you don't install a web browser.
I guess someday if I ever get a Raspberry Pi, it would be fun to make Astroblahhh Desktop work well on that.
I wonder if OrientDB can run on a Raspberry Pi? I guess if Java can, then maybe OrientDB can too?
I made a simple custom PHP error handler to make it block the warnings about the deprecated "mysql" PHP functions. Future not-yet-released extensively renovated versions of Astroblahhh Desktop will (or already do) instead use PHP's newer, recommended PDO database-handling features.
The custom error handler also usually changes the background and text colors of pages containing errors, to make it more obvious at a glance (and pretty) when errors happen.
Once I figure that out, it'll be a lot easier to move forward with all the different projects I have in mind.
Though I made some decent progress with that goal, it's not perfectly achieved for a number of reasons.
My current workaround is to just use new databases and avoid editing my old databases. Astroblahhh Desktop's "copyrecord" and "exportrecord" commands (even in the released versions of Astroblahhh Desktop) work pretty well - so, in the future, once I have everything converted to UTF-8, merging my newer databases with my old databases shouldn't be too troublesome.
More ways Astroblahhh Desktop isn't totally Linuxified yet include the fact that the file organizer still pays no attention to file permissions, file owners, and file groups. The file organizer also still assumes that your filesystem is case insensitive, as it is in Windows.
Other than all that, the file organizer (at least when only reading files) seems to work fairly well natively in Linux, instead of running inside a Windows XP VirtualBox.
But, back to Astroblahhh Desktop. It was pleasant to find that the SHA-512 checksum features are much faster when running ABDesktop in Linux rather than having ABDesktop inside a VirtualBox. I don't know why, but accessing files outside of a VirtualBox from within VirtualBox tends to be noticeably slower than accessing files normally. (That's why I used to store my databases inside my VirtualBox. I tried storing them outside, but searches that ordinarily would take a few seconds at most were taking something like 20 seconds. At least with VirtualBox 4.1.8.)
I love VirtualBox, but having to use VirtualBox always felt like a ball and chain, since VirtualBox is not at all convenient on computers where I don't have a hard drive, and don't feel like moving an external hard drive to it, or where I only have 1 or 2 GB of RAM, or even less.
Even with the latest versions, Astroblahhh Desktop alone (without VirtualBox) shouldn't actually require a powerful computer. The original versions ran surprisingly nicely even on a quite pathetic 1.10 GHz Windows XP computer with only 256 MB RAM. And, I'm not sure, but I think probably the later versions probably still could as well.
Happily, very little had to be done to make Astroblahhh Desktop 2.4 work with PHP 5.6.13.
Probably my next top goal (at least with Astroblahhh Desktop) is to figure out how to convert everything to Unicode UTF-8 encoding.
| Updated the "Internet of Things" post a bit |
|
Tuesday, October 20th, 2015 18:37:29 GMT |
I've been updating this old post from July 12, 2015 from time to time:
Hopefully someday, when I have the CMS (content management system, for lack of a better term) of my dreams, there will be an automatically-updated list of recent updates, maybe kind of like Wikipedia's Recent changes page.