Wednesday, December 16, 2020

Containers over systemd

“Systemd will solve all your problems,” they said.

Having used a number of systemd’s security features to configure a service, I am beginning to suspect everyone uses containers because container runtimes are trying to be secure already.

It's possible to improve the security of a service with systemd, of course.  I’ve worked hard at it.  But in the end, over half the *.service file is consumed with “trying to build my own container out of systemd directives.”  ProtectHome, ProtectSystem, ProtectKernelTunables, Protect This, Protect That, blah blah blah.  The process starts from insecure by default, and then asks me to layer on every individual protection.  This is exactly the sort of thing Linux zealots used to yell at Microsoft about.  ¯\_(ツ)_/¯

But I digress.  I ended up with an excessively long systemd service configuration file, and to apply that to any other service, there’s no option besides copying and pasting those directives.  With every release of systemd, I have to comb the man pages again to see what else is available now, and carefully apply that to every service file.  It’s not easy to tell whether the security posture is up-to-date when the policy is so verbose.

[Updated 2022-03-19: systemd-analyze security foo.service is your friend. This is the best way to get a list of everything systemd thinks about security, and whether it is applied to the unit. It's a little less bad than I thought, but it is still fundamentally the Default Permit and Enumerating Badness approaches.]

Whereas a container has an isolated filesystem (its image) already, so whole classes of configuration (ProtectHome, ProtectSystem, TemporaryFileSystem) become irrelevant.  On top of that, container runtimes start with a more limited set of privileges by default, instead of handing out CAP_SYS_ADMIN and leaving it up to the administrator to carefully disable it.  Escaping from the container runtime is considered a vulnerability; escaping from a poorly-secured systemd service is considered user error.

This is all orthogonal to “containers are interop”, but I think both forces are feeding the containerization craze.  I’m left with the feeling again that systemd should have been the “obvious correct choice,” except they decided usability didn’t matter.

Sunday, December 13, 2020

The world is turning upside-down (2006)

Editor's Note: this is a post from my old Serendipity weblog, "Pawprints of the Mind," which was originally posted nearly 14 years ago, on 2006-12-29. Since then, I can't help but notice that social media created itself on a Pull model—follow who you want—and then replaced it with algorithms to Push into that stream. The text below is reproduced verbatim from the original post.

In the beginning, Push dominated.

A company built a product, and it was Pushed to market. Long ago, newspapers pushed by paper boys on the street carried advertisements. Direct-mail catalogs, pushed through the postal service, were nothing but one long and comprehensive advertisement for the company who created it. The rise of radio and television, both one-way broadcast media, allowed advertisements to be pushed to millions at a time, quickly and easily. Movies and music are produced and pushed, and the producers hope they break even.

After a company pushed their product, they spent plenty of money watching what happened to the sales. Was it going to explode or tank? Was the initial 10,000 piece production run going to be liquidated, or was another run ten times the size waiting in the wings? There was no way to sense demand except to Push supply and watch what happened.

Inevitably, Push was brought to the new media: the Internet. Building on the ideas of direct mail, email lists formed. Spam happened. Spam filters happened. Better spam filters happened. But those may be only a temporary solution.

As Pushing got cheaper, more was pushed, until users fled the deluge. Too much yang leaves people wanting yin. Onto this stage stepped Pull.

Nobody knew it was Pull yet. It called itself RSS or reddit, and it was about users coming and getting it. No more HTML-heavy, graphic-packed email stuffed into their Inbox every other day to make up for the poor quality of the site's search capabilities. No more "Insert-Brand-Here Loyalty Updates". No more subscriptions, passwords, bounce processing, and unsubscribing. No more spam, because the provider no longer needs to know where to send anything; they just wait for users to come and get it.

Pull is about user control. Pull is about saying "I want that" and not having some gatekeeper in the way, trying to extract monopoly rents. This is what scares the recording industry; their value as gatekeepers is plunging as alternative ways of connecting bands and fans arise.

Sunday, December 6, 2020

Can Micropayments Even Work? (2007)

Editor's Note: this is a post from my old Serendipity weblog, "Pawprints of the Mind," which was originally posted over 13 years ago, on 2007-05-13. The text below is reproduced verbatim from the original post.

(This is not entirely academic, as a current goal of my day job essentially amounts to implementing a micropayment system.)

I am beginning to believe that the fundamental problem behind micropayments as a viable option for widespread payment is that credit cards are effectively already micropayments. We're just spoiled by cash. Physical currency is limited by reality. There's not a limitless supply to steal, nor can it be readily created. Undetectable counterfeits cannot be manufactured by poking a few bits inside a computer. Rather, it's difficult to produce high-enough quality counterfeits, which is why good counterfeits only come in $100 notes. The run-of-the-mill counterfeiters are stuck trying to figure out how to make a passable $20 out of card stock or ordinary paper, because anything bigger is subject to too much scrutiny for their materials. (Even then, a local supermarket tests all those, pushing the bar down to $10.)

In essence, I suspect the cost of doing business with a credit card company is mostly the cost of implementing imaginary money securely. The more credit processing costs, the fewer shops join in, and the less of the currency marketshare the creditor ends up with. On the other hand, the services can't be priced so low as to be unprofitable. Not to mention, the more that people use their cards, the more interest the creditor can collect at little extra cost, as all the billing and accounting framework was already in place for that cardholder anyway. Charging less makes economic sense for them, even if they were a monopoly.

A credit card company ends up shaving a few percent off transactions made through them. Micropayments want to be the same thing, only smaller: shave a few percent off penny-sized transactions and make up for it by volume. But the micropayment competes with the credit gateways, if one of the main ways of getting money into the system is to purchase microcurrency on a credit card. Inside the system, the shaving has to be high enough to make up for the transaction cost of the microcurrency being bought and sold, as well as the real costs of doing the transaction and turning a profit.

And if micropayments are essentially equivalent to real currency, then they're also equivalently desirable for fraud, stealing, and counterfeiting: something the large creditors are spending plenty of money on for the best and brightest to counteract. This brings up another point: micropayments probably won't have the same amount of consumer trust as credit cards, because personal liability is legally limited to $50 on the cards. This is not the case for micropayments, which is going to make people not want to have too many of them at one time. That in turn limits the total amount of microcurrency that can be circulated, and restricts the market for higher-priced microsales.

Is it possible to best Visa and MasterCard at their own game?

Sunday, November 29, 2020

Discontinuous Complexity (2007)

Editor's Note: this is a post from my old Serendipity weblog, "Pawprints of the Mind," which was originally posted over 13 years ago, on 2007-10-24. Notably, this was written before I knew about jQuery, which improved on Prototype, eclipsed it entirely, and has since fallen out of fashion. The text below is reproduced verbatim from the original post.

When a system gets sufficiently large, changes become surprisingly difficult. It's as if a gremlin gets into the system, and then perfectly reasonable estimates end up being only halfway to the mark. Or less. This annoys managers, who made a bad decision because the estimate turned out bad, and it annoys engineers because they know their estimate was sensible. Why does this happen?

Slowing Down

I think the key factor is that the system exceeds the size of an individual developer's working memory. In the same way that virtual memory is a lot slower than real memory, development slows down considerably when it exceeds the mind. Tracking work on paper is not instantaneous, and the average developers' choice is to just forget stuff instead. Not that anyone can know when they've forgotten something, or else it wouldn't be forgotten.

The problem with the just-forget method is that it makes coding a lot more time-consuming. You end up writing the same thing, several times, each time accounting for a new layer of information which was forgotten, but later rediscovered. After so much work, you think the code must be beautiful and perfect, until you run it. Another layer of forgotten information becomes apparent, but this time, it has to be painstakingly rediscovered through debugging. There could be several more debug cycles, and if you're unlucky, they can trigger another redesign.

Paper is no panacea either; besides its slowness, it seems to be impossible to track all your thoughts, or sort the relevant from irrelevant. There's nothing like getting halfway through a paper design and then realizing one key detail was missing, and a fresh design cycle must begin. If you're unlucky, there's still a key detail missing.

This overflow is what makes the change so abrupt. There's a sudden, discontinuous jump downward in speed because the system passes a critical point where it's too big to track. Normal development activity has to be rerouted to deal with all the work that has to be done to make "small" changes to the code, and it becomes a huge drain on speed, productivity, and morale. It's no fun to work obviously beyond our capabilities, and the loss of productivity means the speed of accomplishments (and their associated rewards) diminishes as well.

Anticipation

If development must slow when an application reaches a certain size, is there something we can do to stop it from becoming so large in the first place? Could we see this complexity barrier coming, and try to avoid it?

I'm not sure such a thing is possible. Stopping development to go through a shrink phase when the critical point is approaching would require us to be able to see that point before it arrives. The problem is that complexity is easier to manage as it builds up slowly. It's not until some amount of forgetting has happened that we are confronted with the full complexity.

Also, the tendency to break a system down into components or subsystems, and assign separate people to those systems, allows the complexity of the system as a whole to run far ahead of the individual subsystems. By the time we realize the subsystems are out of hand, the whole is practically tangled beyond repair. Besides, your manager probably doesn't want to spend time repairing it even if it wasn't that big.

Familiar Conclusions

No matter what angle I try to approach improving my programming skill from, I keep arriving at the same basic conclusion: that the best systems involve some sort of core, controlled by a separate scripting or extension language. The oldest success of this approach that I know of in common use today is Emacs, which embeds a Lisp dialect for scripting. Having actually made a script for Vim, I have to say that using a ready-made language beats hacking together your own across a handful of major revisions of your program.

I've really begun to see the wisdom in Steve Yegge's viewpoint that HTML is the assembly language of the Web. With SGML languages, document and markup are mixed together, and most of the HTML template you coded up is basically structural support for the actual template data. Even in a template-oriented language like PHP or Smarty™ built on top of HTML, you're forever writing looping code and individual table cells. With a higher-level markup language, you could conceivably just ask for a table, and have it worry about all the exact row/cell markup.

The other major option to reduce the complexity of Web applications, which has apparently been enthusiastically received already, is to put libraries into the languages we have to make them more concise for the actual programming we do. One obvious effort on that front is Prototype, which smooths over (most) browser incompatibilities and adds a number of convenient ways to interact with the Javascript environment. Prototype-based scripts are barely recognizable to the average JS programmer. At what point does a library become an embedded language, if that's even a useful distinction?

In the end, understandable programs come from small, isolated parts. Pushing all variables in the known universe into a template one-by-one is not as simple as providing an interface that lets the template find out whatever it needs to know. Laying out HTML by hand is not as concise as sending a cleaner language through an HTML generator. (And no, XML is not the answer.) Libraries can help, but sometimes, nothing but a language will do.

Wednesday, November 25, 2020

Where is 'localhost'? (Docker networking)

Building up to the devproxy2 release, I set things up to test in Docker, using a known version of Go.  Nothing worked for quite some time.

Core problem: localhost inside the container is not the host network.  I had to configure it to listen on all addresses (by asking Go to connect to ":8080" without a host address), per this other Stack Overflow answer. If I had thought to check the exact error message client side (curl: empty reply from server), I could have solved this one first, not last.

Once I fixed the core problem, everything started working, and I didn’t go back and test the other things I had tried.  So everything else is only a potential problem.

Potential problem: localhost inside the container is not the host network.  I configured it to connect to host.docker.internal instead, per this Stack Overflow answer.  This is the same as the core problem, but in the other direction.

Potential problem: according to my notes, the default network bridge in Docker doesn’t have Internet access. Creating a user-defined bridge solves this.  Furthermore, using Docker Compose automatically creates a user-defined bridge for the stack.  By the time I solved the core problem, I was already using Docker Compose for the networking feature.

Sunday, November 22, 2020

Obvious (2009)

Editor's Note: this is a post from my old Serendipity weblog, "Pawprints of the Mind," which was originally posted almost 12 years ago, on 2009-04-28. The text below is reproduced verbatim from the original post.

When things like Amazon's 1-click patent come to light, suddenly there's a mob of Slashdotters decrying the idea as 'obvious'. My thoughts on the rsync protocol were the same: knowing that there was a program to transfer changed parts of files over the network, I turned my mind to figuring out how the changed parts could be detected. Later I came across a description of the actual rsync protocol, and it was indeed fairly close to my idea for it. Therefore rsync is obvious.

Or is it? The solution may be relatively obvious, but the problem itself was not something that ever crossed my mind before running into the program that solves it. The invisibility of the problem ends up hiding the solution as well.

Apple seems to be actively searching for non-obvious problems: the ones people don't even think of as problems until the iWhatever comes out and redefines the entire market for whatevers. The iPod's clickwheel seems innovative until you realize it's basically a mousewheel. An analog interface to digital systems. Apple only put that kind of interface on because they happened to see that purely binary controls on existing MP3 players were fairly primitive. Once the iPod was released, nobody wanted to admit to being blind to the problems Apple tackled, so their chosen solution (undoubtedly one of many obvious possibilities) was hailed as pure genius.

It seems that the magic is not in who you are, it's in what you choose to think about. If you've never asked, "How quick can we make it for customers to order?" then you'll never end up with 1-click shopping.

Wednesday, November 18, 2020

Using zerofree to reduce Ubuntu/Debian OVA file size

Editor's note [2020-11-21]: this is an update to a 2014 post on the same topic.  Since then, the mount command is also required.

When exporting an OVA file from VirtualBox's "Export Appliance" feature, any disk block that isn't filled with 0 has to be included in the resulting file.  Normally, an OS doesn't create those blocks as it works; when a file is deleted, the space is marked as free, but retains its original contents until it is re-allocated for another file.  If there are a lot of files deleted, a significant amount of non-empty "empty" space can accumulate.

Running the zerofree program writes data of value 0 over all space marked as free, allowing that free space to be excluded from the OVA file again.

The following applies to Ubuntu 18.04 (bionic) and 20.04 (focal) at least; I forget exactly how far back it goes, but the way to get at a command line changed at some point, and there was a period where poweroff didn't actually work in the pre-boot environment.

Regardless, here's how to run zerofree on Debian, Ubuntu, or derivatives.

Sunday, November 15, 2020

State of the Linux Desktop (2009)

Editor's Note: this is a post from my old Serendipity weblog, "Pawprints of the Mind," which was originally posted nearly 12 years ago, on 2009-05-18.  History since then has unfolded differently in some details, but the broad strokes seem to be accurate.  The text below is reproduced verbatim from the original post.

The Linux desktop is a curious thing. Years ago, the annual prediction was that that year was finally going to be the Year of the Linux Desktop, and Windows would be kicked out of market dominance for good. World domination by open source, and all that.

Linux has come a long way since then. Monitors can be largely auto-configured, fonts can be anti-aliased, CDs can be burned without fiddling with ide-scsi, USB works without having to care about your kernel version, and even media keyboards and mice with tilt-wheels seem to work these days.

But the "desktop" has come a long way since then, too. Various OS X versions introduced technologies like Display PDF, Quartz Extreme, Exposé, and other things that Windows Vista copied with WPF and Aero. Linux eventually duplicated them in Compiz, Beryl, Compiz-Fusion, or something. Somewhere in there. It kinda-sorta works now, if you don't want thumbnails of minimized windows.

Now that Linux has joined the current decade, Microsoft is readying Windows 7, with features like thumbnails of windows on the taskbar (with Aero), the 'recent documents' of an application available from the taskbar button, and so forth. Apple certainly has their own plans. And both of them have decent sound and MIDI support, where starting up one audio application is actually not likely to silence another's sound.

So, the "Linux Desktop" is still behind, and always has been. One problem here seems to be that it's not about innovation. It's about copying everyone else's modern technology with ancient components. It's about solving problems again that already have better solutions in other systems, just to see how closely Linux can approximate them with its decades-old core design.

And then someone, or maybe a few people, whine about the solution and come up with an alternative. The alternatives vie for dominance, and the rest of the desktop world moves on while the Linux landscape settles into one predominant choice, if they're lucky. If they're unlucky, you get things like the KDE/Gnome situation.

Due to Linux's philosophy about neither inventing wheels nor including batteries, there's a large number of components with their own maintenance and release schedules, and most of them target "current" versions of other components. For any given complex program, there's a relatively small window of time between the components it uses being released, and becoming obsolete with future releases. Every time components break compatibility, a new version of all programs dependent on them have to be released to deal with the changes. The more changes dealt with, the more complex the program becomes, so compatibility with older versions also gets cut out after a time.

And sometimes, a sweeping change comes through, and no compatibility is maintained between the two versions: App 2.0 will run on Foo 3.0 and newer, and App 1.4 will only run on Foo 2.5 and below.

From a software developer's point of view, then, supporting Linux is a risky proposition. You have to either watch the work in progress on the entire stack, and guess what will be completed and be important to your application, or you end up being taken by surprise when one of your components updates and your application doesn't run on an ever-growing range of Linux systems until you have time to update it.

Apple and Microsoft may have their own problems, but they have the clout and the guts to say, "We need to migrate to doing things this way, and it'll be good for at least 5 years." Look at Windows 95–Me, or NT4–XP. Whereas the rate of change in Linux effectively limits compatibility to about a year at the most. So most serious software gets developed for other platforms, and most serious users are on those platforms as well. Then familiarity begets desire for those platforms over Linux.

In short, Linux is sort of the Perl of operating systems: messy and aging. People working on Linux desktop technologies seem to be doing it to prove that they can. And secretly, they hope to get horses fast enough to put that Ford guy out of business.

Saturday, October 24, 2020

Defense in Depth: Read-Only Document Root

One security rule I live by as a sysadmin, which admittedly causes me a lot of trouble with PHP apps, is that the running web server user is never able to write to files within the document root.

Any user-file upload/download mechanism goes somewhere else, so that read access can do a permission check, and be delivered through readfile() or a redirect to a CDN, not passed through a complex mechanism that may decide to execute the file's contents.

Configuration files are similarly locked down.  A project that configures itself with PHP code is a project that offers arbitrary code execution to anyone who can change that configuration file.

I don't let the web user install new code to /bin; we shouldn't let the web user install new code to /srv/www. It's the server, not a system administrator.

Not coincidentally, our document root is never the root of a git repository.  It's typically placed in a public folder, allowing us to have a special writable folder like cache both within the project/deployment, and outside of the document root.

Saturday, October 17, 2020

MySQL on MacOS: Discoveries

For a long time, I’ve installed MySQL within my development virtual machine (VM), loaded it with a minimal subset of production data, and pointed the database hostname to 127.0.0.1 through the VM’s /etc/hosts file.

However, working from home, I realized that the my slower DSL connection did not affect the amount of time it took to pull a fresh subset of production data into the VM.

I finally went to the trouble of installing MySQL on the host side, and configuring everything to work with that, so that filling it does not require the data to cross any hypervisor boundaries at all.  And I learned some things!

  1. The installer sets it up to launch as _mysql user
  2. Using the wrong CA certificate causes padding errors
  3. MySQL 8.0.14+ allows a separate "administrative management" port
  4. MySQL’s AES_ENCRYPT() function makes the worst possible choice by default
  5. The speedup is dominated by MySQL's disk access

Tuesday, September 8, 2020

(Not) Learning Rust

I want Rust to succeed.  It seems like the best gambit for fixing C, where the slightest mistake is likely to be a security hole… or to be turned into one by a newer compiler version.

On the other hand, actually learning it has proven extremely difficult.  Very little of Rust is explained in terms of something else; and all of the basic mechanics of the language feel weird.

After the Rust Book and the Cookbook and maybe some other stuff from Awesome Rust, I went through the tutorial about writing very bad linked lists; there were places that were like, “Can we do X? [code that does X].”  I would stop there, run that code, and try to solve the error myself.  I never could. I would read on, and it would be, “The error tells us we need to call F” so I’d try to figure out how to call F.  I never could figure it out, even with that hint. And I still didn’t have the capability to go from error message to even discovering F, let alone understanding that F would apply to the error.

I threw myself at the wall several times, and failed to advance past the part where one writes a number-guessing game for the CLI.

Moreover, I don’t have a motivating project to “learn Rust” with.  It’s mostly an exercise in following other people’s code, breaking it, or trying to improve it, and ending up with incomprehensible and unactionable errors.  The compiler errors try to be helpful, but if the code starts out far enough off the rails, it’s just flailing, same as me.

When a survey comes in that says “Opinions of Rust are very favorable, but people aren’t using it,” I understand.  I understand completely.  We all want something to fix entire bug classes, but keep the bare-metal performance, and Rust is basically without competition there.  But we can’t all devote enough time and energy to learning Rust from scratch.

Tuesday, August 25, 2020

Python has descended into chaos

At my employer, Python is not our primary language; the main system was built in Perl and is now a Perl/PHP hybrid. The surrounding systems have been built/replaced with PHP, but the core code is resistant to change. Management doesn't want to mess with the part where the actual money flows in.

For seamless session handling—local memcached on dev machines, but shared storage in production—we have memcache-dynamo. It started life in Perl. I rewrote it in Python, as part of the quest to replace all Perl with something more popular in modern times. (I figured async/await syntax would be easier for future developers than React-PHP.) It's been working fine on Ubuntu 18.04 for some time, but then I wanted to update to Ubuntu 20.04.

It quickly turned irritating. The Python 3.6 dependencies didn't work on 3.8. It turns out that the updated dependencies for 3.8 don't work on 3.6, either. I guess this explains why pipenv is so insistent that "requires 3.6" means "exactly 3.6".

Python 3 isn't just a version with some backward-incompatible changes; it's fundamentally a new culture with new values. And honestly, that probably means less Python in my future. I don't want to promote another language into the main systems, and Python no longer seems to be suitable for fire-and-forget background services.

(In the end, I wrote a new layer on our installer that checks the Python version, then copies e.g. Pipfile-3.8 and Pipfile.lock-3.8 into their standard location before starting pipenv. It's a horrible mess and I hate it so much. Please, don't start a Python 3 transition every year!)

Saturday, August 15, 2020

Black Lives Matter

This is a very focused blog.  One of the few comments I received on my old s9y blog was, “The technical content is great, but it could use fewer political and religious diatribes.”

We don’t exist outside of these social, political, or economic systems, though.  We are not separate from our own unconscious and its biases.  At the end of the day, if I truly believe in equality, in liberty and justice for all, then I must be willing to work for that.

The other crucial problem is that my silence will be interpreted as assent, which is a serious problem when my silence is interpreted as supporting something I deeply oppose.

Black lives matter.  Black quality of life matters.

Defunding and demilitarizing the police is an excellent first step.  But actions speak louder than words, and we really have to change those actions as a society; we have to start believing that petty crime does not justify lethal force, and genuinely prosecute those who act otherwise.  We should find it normal for police to de-escalate a situation.

And police reform (at a level that genuinely impacts the statistics and improves the lives of actual people) itself is only the first step.  Defunding the prison system and releasing non-violent offenders who were swept up in the war on drugs is overdue as well.  This is a problem we made ourselves, and our cultural narrative about it drives it toward being a self-fulfilling prophecy.

What’s next after that?  I don’t know.  That’s where listening to other people is important.  All I can guarantee is that we will accomplish nothing if we deflect or ignore the topic entirely.

Further food for thought:

Saturday, August 8, 2020

On "Perl 7"

Previous history:

  1. SawyerX: Announcing Perl 7.
  2. Grinnz: Perl 7: A Risk-Benefit Analysis.
  3. RJBS: On Perl 7 and the Perl Steering Comittee.

My comments at this point:

Hey #perl core. Is there anywhere I can upload a cpanfile, so you can know what's important in production? Like Debian's popularity-contest, but specialized. (We install most everything via Ubuntu repositories, so you don't get downloads/test feedback.)

The perl7 thing is probably accelerating our plans to phase Perl out entirely. I'm not sure what we'll do if Ubuntu 22.04 ships a "Perl 7" that can't run Perl 5 code.

To be honest, ~20% of hits to our client-facing site are served by CGI.pm because of some failures on my part (should have used CGI::Emulate::PSGI?) and on management (50% time savings is great, but let's work on features... forever.)

Anyway, it's going to be real hard to sell "work on Perl7 support" to a management team who have been ramping up the priority on "rewrite the application."

(Lightly edited from my twitter thread.)

Now, I realize I'm nobody, and nobody has to listen to me, but I think the popularity-ranking thing is kind of important, and maybe even a "State of Perl" survey to find out what's even broken. All I know from my vantage point is that Perl looks fairly dead already, despite how much I loved it. The Perl 7 crisis has so far been more anxiety-inducing than inspiring.

Updated 2022-03-19: Perl 7 is basically dead? Business as usual? I still can't help illuminate the DarkPAN for them, and management is inching closer to sunsetting our own Perl code. RIP.

Thursday, April 30, 2020

pipenv's Surprise

Warning: Python 3.6 was not found on your system
You can specify specific versions of Python with:
  $ pipenv --python path/to/python
I am left with no clear path to making this project run with the system Python 3 across just two versions of Ubuntu LTS.  It doesn't work on Focal (Python 3.8) as-is, and if I update the Pipfile.lock, it won't run on Bionic (Python 3.6).  It doesn't have shebangs for python3.6, as it expects to run on Python 3.6 or up.  This is how SemVer works!

Maybe the answer is to build my own tools in order to run this in a way that suits me. Which is: I really want a build phase to create a tarball, which can be extracted and run in-place later. All the complexity of vendoring should be finished by the time deployment (from tarball) occurs, for reliability and reproducibility.

I do not want to write this project a third time, probably in a language I hate, just because virtualenv is both wholly inadequate, and central to Python tooling.

(Something like the way the awscli-bundle is packaged is an interesting start, but it still has to grind through a lot of code to "install" itself. It's not unzip-and-go. Also, I've got no idea how they build that in the first place.)

Wednesday, March 11, 2020

Unexpected Noise in Ardour

I've been learning to mix with Ubuntu Studio 19.10 and Ardour5, but I've had a puzzling issue: noise.

My workflow has been to load in the .wav files I want to mix, and then start working, but it always had some low-level noise on the channels.  Even when Ardour was not supposed to be playing, the noise would be there.

It turns out that Ardour still connects the inputs to the track, and passes them through the effects chain.  This is why the noise responds to the fader, noise gate effects, and the like.  The solution is to go to the input on each mixer strip and switch it from 1 (or 2, or 1/2, etc.) to "Disconnected" instead.  Otherwise, at minimum, thermal noise gets added on the channel.

I happened to discover this through a pair of accidents: I had a cheap headset mic connected and noise was worse than ever, but also, all of the meters jumped when some background noise happened.  Disconnecting the inputs fixed the noise, without affecting the ability to play back the imported audio.

(This isn't the normal fare for my blog, but if I put it on tumblr instead, it'll never be seen again, and what good is that?)