Celandine Part 2: Sweet Heavens This Is Difficult Let’s Just Copy Someone Else’s Method

Hiatus? What hiatus?

Previous

So, when we last left Celandine, the Julia-based computational chemistry diversion, I was stuck looking through PyQuante and wondering how it works when it seems like it shouldn’t. Since then:

All of these are excellent resources, huge thanks to their creators. If, heaven help you, you are following along with this in order to improve your own understanding, I would suggest having a crack at the worked example project in the last link before looking through the “solution” below.

Continue reading “Celandine Part 2: Sweet Heavens This Is Difficult Let’s Just Copy Someone Else’s Method”

Advertisements
Celandine Part 2: Sweet Heavens This Is Difficult Let’s Just Copy Someone Else’s Method

A series of somewhat-negative thoughts not individually deserving blog posts of their own

It annoys me when people come up with evolutionary psychology “Just-So stories” to justify their politics. Well, obviously. Because it’s really annoying. But there’s a smaller mistake hiding inside the larger one, which is that when they start talking about the Ancestral Environment, it’s always the same sort of Tribal Savannah thing. Or if they’re big into HBD, a very similar Tribal Deciduous Forest kind of thing. But actually, we should expect most evolution-shaped psychological features to pre-date humans entirely, in much the same way pretty much every other biological thing did. Parental instincts, group dynamics, even apparently cognitive biases (so cool). It all came before humans, and if your Just-So story begins “hunter-gatherers would have…” you can know immediately that it’s probably not how things actually happened, just because of that.

(If your story would have begun “tribes in Africa would have…”, it is beyond saving).

((The whole subject area is thronging with transparent attempts to revive teleological arguments for why Liking Things I Don’t Like is objectively wrong, but now with a scientific rather than religious gloss. Sigh.))


There is fundamentally no way to reconcile collectivism and individualism. It’s the simplest possible clash of terminal values. This is what some people are grasping towards when they advocate that we “bash the fash” – that fundamentally it’s impossible for collectivists to be happy in an individualist society and vice-versa. The increasing popularity of collectivist ideologies leaves individualists with the choice of being miserable or making others miserable. Or dead. Everyone (well, everyone as smart or smarter than me that I’ve discussed this with or seen writing about it – sample size ~5) invents Archipelago as an attempt to solve this, but notice how even Scott, possibly the smartest of the sample, has to sneak “but fundamentally individualist” in at the lowest level. Individuals can choose to join collectivist groups or leave them – not a solution that is acceptable to collectivists!

This is a facet of the fundamental difficulty of creating a Good World. I do not think it can be resolved for humans as they currently are, let alone by them.


Doki Doki Literature Club would be decent if it didn’t warn you so strenuously about being Spooky. As-is, it just comes across as trying too hard. It’s quite unfortunate, because some decent ideas are buried in there among with the gimmicks and the necessities of playing along with the conventions of a terrible genre. For instance, the antagonist(?)’s total lack of personality beyond “totally loves the player character” is definitely making a point about dating sims in general (the fact that people like her says much the same thing and to much the same effect as people liking Asuka and Rei). But ultimately, it spends too much time knocking on your skull and asking if you’re scared yet, putting up flashing neon signs around all the foreshadowing, and sort of nudging the audience in an expectant way like “hey that was a cool trick I just pulled, right?” While I continue to strongly support the idea of content warnings, this really highlighted that they need to be “need-to-know” information, something you wouldn’t see if you’re not looking actively.


Keats wrote about “Negative Capability.” Normally I hate this kind of thing – logical consistency is very important to me in fiction and ‘just turn your brain off’ is not an acceptable answer to such criticism, mysterious answers are a contradiction in terms, handwaving can’t save metarationality, that kind of thing. But when it comes to them chinese cartoons for girls, a small handful put me into this state so effortlessly and unobjectionably I didn’t even notice until now – “Girls’ Last Tour” was what made me notice the experience, but looking back it’s a common factor between a lot of things I’ve loved, like Mushishi, Kino’s Journey, and all-time favourite Haibane Renmei. The defining emotional feature of such works is usually “mono no aware,” though, another kind of mystery-feeling that’s very hard to describe. It’s a little hard to say whether these are attempts to grasp after the same concept, since mental states are so horribly non-transferable, but they might be. The upshot being that a tranquil awareness of mystery: actually quite pleasant if you can achieve it.


What would a philosophical Experience Machine that provides the experience of “exiting the experience-machine that is our reality” (as in the films) be like? People want to have an effect on “actual reality,” which is why they reject EMs, but what makes reality, reality? If you’re too desperate to avoid being caught in simulation, do you end up too easily tricked into entering Matrices that are labelled “Matrix Exit”?

I feel like there’s an important principle here. Natalie Ferno’s Reality Is Where You’re Standing Principle. Something like that.


From my perspective, the America I see in things like Stranger Things (or Life is Strange, or etc etc) exists only in that context. As far as I’m concerned, it’s not depicting a Normal Small Town with weird stuff happening – it’s depicting that bizarre almost-modern fantasy world where weird stuff happens all the time. This rather dampens the effect, I think?

Or maybe it’s the other way around, and actually it seems that way to most people in the USA too. And it’s the Normal Small Town people who have the weird experience of perceiving the fundamental wrongness of what’s happening. That would seem unlikely, though. Surely that’s the feeling you want most of the audience to have?

Figure/Ground effects.


I’m honestly pretty proud of the discussion around the Lootbox Issue. I was expecting a lot of “nuh uh, it’s not gambling because you always get something,” and while I’ve seen that a great many times in the context of “legally it’s not considered gambling because…,” mostly people seem to have a good grasp of what the central aspects of each are and how they’re the same. Maybe people are finally developing cognitive tools against the noncentral fallacy and relatives?


People almost all have a sense of the Zeitgeist, a Zeitgeistbewusstsein perhaps (I do not speak German). A feel that “these days everything’s so PC” or “the far-right is becoming normalised” or “NASA and the CIA are tricking everyone into thinking the world is round” (explaining how this third belief is of the same type as the first two might be a bit tricky, so I’ll elide doing so). These are, I am more and more convinced, almost universally wrong. The whole notion of a Zeitgeist increasingly strikes me as purely a reification of the availability heuristic. Good for spinning stories out of, but not a useful factor in any real, predictive explanation of anything.

A series of somewhat-negative thoughts not individually deserving blog posts of their own

Trails in the Sky

Minor spoilers only.

It has become common, even cliche, to say that games should be about the mechanical element, the “gameplay aspect.” For instance:

Well then, dear game creators: Play to your strengths! You can do much more than merely aping movies. Maybe concentrate on your gameplay for example. Trust in your audience being intelligent enough to transfer abstract outcomes into emergent stories and learn from those. You might hand them a few incentives in the form of dynamically integrated narrative legos on the way. And even if you want to present a self-contained narrative, always incorporate your players. Take them seriously and do not be afraid of challenging them. Don’t just use your engine as a meaningless mediator, but as a tool of collaborative storytelling between author and recipient. After all, that is what makes your medium special and grants it its unique potential.

You’d get the impression that nothing is worse than a game that is a ‘film with playable parts,’ so to speak. That telling a simple-ish story in the obvious way is doing something wrong, is aping another medium rather than developing a true “video game style” of storytelling.

(Alright, let’s be clear that this a big old debate with a lot of arguments on both sides. I’m cherry-picking one person making an actually quite reasonable and balanced point, as a framing device. But also because I can’t remember the last time I heard someone ask for more narrative-driven games. Saying gameplay is more important just seems like the cooler position for smart media critics)

On an intellectual level, I agree with all this. It’s hard to argue that games should be something other than games, after all. But fundamentally, my heart belongs to narrative. Some games weave their story into the game itself in rich and compelling ways, and those are wonderful, but a lot of games – particularly JRPGs – go more for “series of plot-advancing cutscenes mixed up with plot-irrelevant gameplay.” The Legend of Heroes: Trails in the Sky is one of this category, mixing a turn-and-grid-based RPG (one can hardly call it a tactical RPG with a party size of 4, but that would be the right way to think about it) with a narrative that ranks among the best in games. And I loved it. Here are some reasons that you could use to judge whether you ought to give it a try.

Slow start: In my opinion, the best fantasy fiction isn’t terribly grim and dark. The Lord of the Rings is one of my absolute favourites, and fans agree that it’s not so much an epic story about kings and battles (such as shown in the films), as a story about wholesome hobbits doing comfy things, occasionally interrupted by a quest to save the world. Trails in the Sky takes this to an extreme degree: the first half of the first game could easily be from the iyashikei genre. Sure, there are events going on, but they fit the scope of “wandering do-gooder private police trainees doing good deeds of minor consequence.” You help with a school play, for crying out loud. It’s just a really pleasant intro to the series that makes it easy to get invested in the characters since they’re not angsting or struggling in some cosmic battle or anything. By drawing it out longer than normal, the game is able to have a larger main cast, a more endearing world, and a stronger central relationship than is common.

Admittedly, this comes at the cost of discouraging people who will be waiting impatiently for the plot to get going. Therefore my advice is: play this if you want a fluffier kind of game – think along the lines of, say, Stardew Valley, but as an RPG.

EstelleDescribesTheGameAgain

Smart characters: There are at least two ways characters can be smart – well, two that I like to see; “his IQ is 400!” and the such are not actually a kind of smartness. The first is to be smartly chosen; Doylian smartness. This is the kind of smartness that made Estelle rather than Joshua the point-of-view character of Trails in the Sky. He’s the more typical JRPG protagonist, but just less fun to spend time with – it’d be harder to maintain the story’s key mysterious elements, and harder to appreciate the optimistic tone, if the protagonist weren’t the character who’d be “the energetic sidekick girl” in any other game. The choice to switch the two feels like a touch of genius, an apparently-small change that cascades to alter the entire game in a positive way.

The second is to have characters that make intelligent decisions. An important part of this is that non-central characters make plans and develop actions off-screen, but the central aspect is not doing stupid things. There are one or two cases where they do, but for the most part, the characters in Trails in the Sky make sensible decisions, are able to make deductions as successfully as the player does, and so forth. They’re also clear on the fact that the in-combat magic is an actually real thing used in pretty much the same way the game presents it, which is nice. Admittedly you do sometimes beat giant robots to death with knives, but hey, JRPG.

Free will ain’t all that: My position on choices in games is that if you can’t offer meaningful choices, you shouldn’t offer meaningless ones. Trails in the Sky doesn’t screw around with any pick-your-hair-style character design, pick-your-waifu romance, or pick-your-palette-swap endings. This doesn’t mean you have no choices. You still have choices to make in the actual gameplay, choices about who to talk to and what sidequests to undertake, that kind of thing. If “meaningful” means “altering the story,” then no, nothing you do matters. But that’s not really the kind of meaning that makes good narrative or good gameplay – a choose-your-own-adventure book lets you determine the ending, but the gameplay is awful. And if you have to make every NPC only connect in any depth to the main character – because they need to be available as a romantic path, or because you don’t know which NPCs the player will interact with, that kind of thing – you miss out on a massive amount of potential interactions that deepen the game world and strengthen immersion.

This is to say, I’m perfectly happy to have a linear story. Multiple paths tend to lack depth for any one path.

Liber’l conspiracy: I’ve said before that I’m not above admitting when I like or dislike the political aspect of a work. And these games were out to pander to me from the very start. Not only that, but it seriously discusses some of the difficulties of such a political position, look:
IWasNotExpectingSODAMNMANYRelevantPoliticalDiscussionsFromThisGame.png
Sure, it could go further, but it’s a game, not a debate. And Estelle can hardly be expected to have as well-developed a political philosophy as a queen, after all.

This is a stark distinction from the typical prelapsarian narrative of fantasy fiction in general. Liberl is a constitutional monarchy, sure, but it actually explains how it’s functionally different from just a monarchy. Compare, say, Final Fantasy XII, where it’s not really clear why the monarchy you’re in is really any better than the monarchy you’re being invaded by. I suppose the reactionary argument would be that you don’t need a reason to love your country except that it’s yours, but tough luck, I do actually.

This isn’t even to say it’s completely unfair to the Empire or the Parliamentary Democracy that comprise the other major political factions involved in the story, though. All the nations’ governance systems seem to have some merits and some drawbacks, and if the game is including some distortion in favour of the point-of-view state, well, that’s only to be expected. Can one have an “unreliable cultural narrator”? In short, the story’s ideas are thought through in a way that’s not all that common, and that I don’t think you could do with sprawling multi-narratives or subtle emergent narratives.

Worldbuilder: This is the really big one, the thing everyone praises the broader Trails series for in general. The world is vivid beyond the scope of most video games to attempt. NPCs that wouldn’t even merit a name in a lot of games have complicated on-going stories; historical events are explained in depth; there are some surprisingly lengthy stories found in books in libraries that are relevant not in some abstract literary sense but in the everyday sense of “people in the world reference having read the books, or having written them.” There are port towns being hollowed out by increasing air trade, side-quests that introduce you to important metaphysical concepts, even more NPC stories to follow. The whole world feels alive and dynamic, as opposed to the “timeless” atmosphere that a lot of fantasy RPGs aim for even when they’re in the middle of similar magi-industrial revolutions.

It’s really hard to describe the sheer scope of it – it’s something that games can do better, even linear narrative-driven games, because you can freely include irrelevant details and side-stories without alienating the audience so much, when people can just ignore it if they prefer. That said, your enjoyment of the game is going to be much higher if you do prefer to explore the world to the fullest – the game isn’t pretending like it’s just as good if you don’t.

ThisGameThoughForReal

Not even a bad game: And you know what? It’s not even like it’s a bad game held up by good writing, for all that the writing is stellar. It’s a good game! The combat mechanics are easy to grasp but allow a lot of depth. Figuring out different magitek configurations to access different spells is surprisingly fun, if somewhat annoying to interface with at times. Party members have substantial mechanical differences, while still giving you plenty of freedom to arrange your strategy as you like. The soundtrack – is that part of the game, or part of the narrative? Either way, it is amazing. The aesthetic style as well works amazingly well for capturing the cosy and the epic alike while keeping it clear what’s going on.

All that said, none of this is unique to this game at all. You can find comparable gameplay elsewhere. But are games about gameplay? Yes, in some sense that’s true. But they have a lot of other stuff going on as well, and it’s important not to get so caught up in horticultural analysis that you forget to smell the roses. Trails in the Sky is without doubt one of my favorite games of all time not because of the okay gameplay, but because of the everything else. Recommended without reservation.

Trails in the Sky

Why’reheading

Why’re we teaching teenagers about safe sexual practice, when we could just try to treat the underlying psychological issues that make them want to have sex?

Wait, no, that’s stupid [1]. Why’re we allowing people to buy make-up or cosmetic surgery, when we could just try to treat the underlying psychological issues that make people unhappy with their appearance?

Wait, that’s stupid [2]. Why’re we supporting people transitioning to the appropriate gender when we could just try to treat the underlying psychological issues that cause dysphoria?

Wait, that’s also stupid [3]. Why’re we using exercise regimes and gastric bands when the solution to obesity is to treat the underlying psychological issues that make people want to be thin?

Wait, still stupid [4]. Why’re we suggesting improvements to societal structure when clearly we could just treat the underlying psychological issues that make people in modern societies unhappy?

That’s probably stupid.

I’ve seen all of those five arguments made in varying contexts. The point of this selection was to maximise the number of people who have at least one issue on each side, so as to frame the discussion more neutrally. But if you’d have guessed that the origin of the discussion was the highly-contentious third, well, I wouldn’t call your guess wrong. But equally, the aim is to skirt around the highly-abstract last one, which is by far the most important.

Seeing these five side-by-side makes it prudent to start by examining the most common argument that’s made in favour of each of them – a slippery slope argument of the form “but if we accept people getting their noses shrunk, how can we say it’s wrong for them to turn themselves into freakish monstrosities of chitin and tentacles that should be cleansed from the world with fire?” Well, we can’t, same as there’s nothing wrong with people removing their limbs if it’ll make them more comfortable with their body, or people trying to find fulfillment in life rather than altering themselves to find their current circumstances fulfilling. Sure, maybe they’d be better off following Buddha’s advice and trying to become perfectly phlegmatic about things, but that’s ultimately a demand for people to change in an unlikely way.

Now, perhaps with sufficient enlightenment or technological advancement, we could make it less unlikely. We could find drugs that treat gender dysphoria or autism or being unhappy about being overweight. That’d certainly be a start. Why not go further? We could have a pill to cure ennui, a surgical procedure to make Mondays seem less horrible, or a vaccine to prevent liking Nickelback! Wait, is this starting to sound familiar?

The well-read will recognize this as an argument fundamentally about wireheading. Well, perhaps it is not such a good mark of entry into the elite once it has a wiki page. The foresighted will note that I am also constructing a slippery slope argument in the opposite direction: if the answer to the question “how much wireheading do we support” is not “none,” or “only as much as people want to do to themselves,” then how much? If people want to be thin, rather than accept being fat and become happy with it, why should we tell them that correcting their psyche is the better option?

This isn’t exactly the same thing as wireheading on every level, of course. But it certainly seems like any argument that proves that it’s right to alter people’s minds in the listed ways ought to be strong enough to also prove that it’s more generally right to alter minds in any number of unlisted ways. And if we take it as given that we don’t want to just alter ourselves to be permanently blissed out by everything, it follows that any particular argument for why certain people should alter themselves to be happy with any particular thing is in need of justification beyond “it would make them happier.”

There are a handful of such justifications that get seen fairly often. Appeals to Nature or God are of little interest to me; if you wish to make such an appeal, I suppose that’s your prerogative. Sometimes people make appeals to unfair competition – “some people are more attractive, and given the demands of competition, this amounts to a kind of tax on unattractiveness, which would be better off removed.” While I kind of agree, pushing back against the existence of options to alter the environment in favour of altering the individual just seems like a poor way to resolve this. If there are any good ways, I don’t know of them, though.

To go back to an earlier point – in Buddhism, there is a story of a farmer with 83 problems [5]. The story goes that the farmer went to see Buddha, who was known to be wise beyond wisdom, to seek counsel on how to rein in his errant son. Buddha said he could not help with that. Well, said the farmer, maybe you could advise me on how to mend my leaky roof. Buddha said he could not help with that. Okay, said the farmer, maybe you could teach me how to mend shoes. Buddha said he could not help with that. And so on through the farmer’s entire list of problems. At the end of the list, the farmer scowled and said “is there anything you can do?” And Buddha said he could solve the farmer’s 84th problem: that the farmer wanted to not have problems.

This must have made Buddha feel extremely clever, except that I’m pretty sure he was above that. But the farmer’s son was still errant; his roof still leaked; his shoes still tattered. One might even argue – given that Buddha could not wirehead anyone, and could only suggest decades of meditation and self-doubt – that he gave the farmer the 84th problem of feeling like caring about all those other problems was his fault for not being phlegmatic enough. Now, maybe the farmer attained enlightenment and was happy – or whatever positive-affect adjective you use to describe enlightened beings, anyway – and if so, good, but did Buddha really do all that he should in this story? Was it okay for him to sit back, content with having provided only the option to remove the perception of there being any problems?

As with the list of examples above, there’s one simple answer: that if someone prefers not to alter their preferences, then we should not say that having provided the option of doing so fulfills all moral obligations to alleviate their suffering. There are lines to be drawn on how far it is acceptable to go in pursuit of such, but the line is not here unless we want to say that in every case listed so far and many others besides the correct approach is “just don’t care about it.”

Well, I hope I’ve explained a bit about why I think wireheading is the wrong sort of approach to the, hah, problem of having an imperfect world. If people want to alter themselves, sure, but the mere existence of that as an option would not be enough to dismiss unhappiness.


[1] – Because that sounds completely impossible.
[2] – Because self-esteem can only get you so far; attractiveness isn’t going to be purely socio-cultural.
[3] – Again, that also sounds pretty much impossible.
[4] – Again, social norms aren’t so loose that people can expect to do equally well by following or by defying them.

The important note here at the foot is this: these arguments are constructed without reference to what the patient wants, i.e. no “but they probably don’t want to be cured of that desire.” And if you’re asking some of the questions but in other cases using answers similar to the footnotes, note that these are pretty interchangeable. For instance, obesity kills as surely as cancer does – and so does gender dysphoria, if you accept that “just have them not commit suicide after we insist they live a life that will make them want to” is not a valid method of engineering solutions, in the same way that “adopt the NAP” is not a useful solution to propose for gun violence.

[5] – It’s not clear why that number in particular, but I did remember the number perfectly despite having heard the story only once. Maybe there’s someting to it!

Why’reheading

Why can’t we skip all these tedious intermediate steps and just succeed already?

Suppose you had a discussion like this:

A: Choristers are terrible! They keep singing things all the time, and it gets on my nerves!
B: Have you tried earplugs?
A: Of course! They’re worthless! Uncomfortable, constantly in need of replacement, hardly block the awful singing but do somehow always make me miss important phone calls…
B: Okay, not that then. Have you tried asking them to stop?
A: Only every day for the last eternity. Why won’t they stop? Argh.
B: Maybe try asking them to sing something different, that you’ll like?
A: I don’t want them to sing something else, I want them to stop.
B: Or maybe you could offer to help them find a more soundproof room somewhere?
A: Why should I help them?! They’re torturing me! Why can’t they just stop doing it?
B: Perhaps some kind of rotating schedule, so you can be elsewhere when they sing…
A: Argh! No! They need to just not do it!

We could definitely accuse B of being unsympathetic. But A is also being unsympathetic, and so are the choristers, and it doesn’t really matter anyway. The point is that B is trying to be pragmatic – find a workable solution that makes A less unhappy. But A doesn’t seem all that interested in the workarounds – their only plan is to hope for the simpler solution of everyone abiding by A’s own preferences.

Let’s briefly consider some real-world examples:

And so on through a hundred other tedious culture-wars-by-proxy, “why can’t people just diet and stick to it,” “why can’t people just have more feminist sexual preferences,” “why can’t people just get jobs,”… All different in their exact causes but all containing a trace of the same error. Now that everyone is at least a little bit angry and considering leaving a comment about how their pet issue is totally different (hey! Just like mine!), we can move on.

Hopefully the idea is now clear. Someone has some extremely precious value like pro-choice, free speech, having guns, etc. That value gets questioned by other people who have different values. The person wishes other people would stop doing that. The problem is that, no matter how important it is (to the requester) that the value be respected, it’s not enough to make people actually do it. And emphasising that importance by repeated injunction does nothing.

Which is to say, there’s a tendency to try to object to a proposed solution by saying “but the real problem is that people are causing a problem. People just need to stop doing that.” Essentially, asking for people to change in an unlikely way as a substitute for discussing the proposed solution on a deeper level and gaining understanding of why it’s not satisfactory that can be used to refine the solution and so on.

I’ve made this mistake over and over again, on issues from environmentalism to electoral reform to foreign policy. It’s ludicrously hard to debate ideas without ever asking for the impossible. It could be seen as a kind of fallacy of perfectionism, but I prefer to think of it as its own thing, a kind of cognitive failure mode based around the fear that one’s values won’t be respected and the tendency to stop looking for a solution once someone else can be blamed.

The objection is obvious: but isn’t asking for less “asking for other people to change in unlikely ways” asking for people to change in an unlikely way? Yes, it kind of is. Therefore, here are some proposed practical workarounds:

  • Express the sentiment as “just to check, we agree that it would be best if … ?” – The aim here is to placate the part of the mind that is worried that the other participants won’t respect your highly regarded value.
  • Emphasize not wanting to be dragged off-topic when mentioning that it would be nice if whatever optimal path could be taken instead of compromising. This seems prone to failure. No amount of “let’s not get off topic, but…” has ever prevented discussions getting off-topic.
  • Resist the temptation to respond to “why can’t X just V?” with disagreement about whether it would be good if X just V. It is sometimes possible to find a way to express the idea that the principle is sound but an unhelpful way of looking at the original question; but if not, you’re usually allowed to just drop the line of discussion.
  • Ignore the discussions themselves. Then, write a long meta-level rant on your blog about it. This solves the problem forever.
Why can’t we skip all these tedious intermediate steps and just succeed already?

Celandine part 1 – Basics and Basis

Current code base available here

“I’d quite like to write a very simple computational chemistry program in Julia,” thought lots of people all the time, “I wonder how I’d go about that.”

Step one is to find out if the problem has been solved before. Not quite, but there is an open-source computational chemistry program in Python. So I’m not so much re-inventing the wheel as re-branding the Rotating Axle-Based Object Transport Circle. Still, it might be fun to try, and more importantly, we can follow the existing methodology as a guide.

It would make sense for step two to be “look up a bunch of sources on how to do computational chemistry,” and indeed this is what I did first. But vague memories of university would have been sufficient to provide the answer: there are a whole bunch of different approaches, and each will need its own algorithmic implementation (you could probably get that much just by guessing, even). However, they will share common features: input format, possible outputs, basis sets, and the such. Therefore this first post will, very conveniently for a lazy blogger with a day job and other hobbies, not in fact contain any computational chemistry. Step two, then, is to implement a basic structure that can read and display molecular geometries.

moleculeReader.jl: module for reading molecular structures from files into arrays
A molecular structure is, fundamentally, a 4-column 2D array, the first column containing atomic number and the other three giving (x,y,z) coordinates

function convertTextBlockToArray(text::Array)
    #Given the subset of text in the file which constructs molecules,
    #convert to an array of integers
    #This creates a 1D array of vectors
    A = [lineConvert(l) for l in text]
    #This converts it to a 2D array
    A2 = permutedims(hcat(A...), [2,1])
    #Obviously there are other ways to reach this result
    return A2
end

To explain how this works: the line creating A uses a comprehension to create a vector of whatever lineconvert is returning. The structure “hcat(A…)” concatenates all the contents of A together into a single 2D array instead. permutedims is Julia’s generalisation of transposition; permutedims(Array, [2,1]) is equivalent to transpose(Array). The result is a 2D array of converted lines from the text. We could also obtain this result by iterating over the array, converting and concatenating each line. But this would look a lot uglier, and since pretty much all reasonable molecules have less than 100 atoms, this slower method is fine.

function lineConvert(line)
    #Converts a single line from string to numerics
    a = split(line, ',')
    b = map(strip, a)
    c = map(x->parse(Float64, x), b)
    return c
end

This can be done in one line of stacked maps, but there’s not really any reason to and it’s very hard to read it that way. We have to use the somewhat ugly lambda expression “x->parse(Float64,x)” rather than just “parse” in order to force parse to treat all numbers as Float64. This is still somewhat annoying since we’d ideally like to treat the atomic numbers as integers, but that would mean using a mixed array. It seems acceptable to use floating-point for atomic numbers here, since they’re not actually used for any arithmetic until much later.

This function could still stand to be improved. Ideally it ought to be possible to freely switch between writing the atomic numbers in the input file, and writing the symbols (H, He, C, Br, etc). And a proper version of this would probably find some way to use integers for atomic numbers.

function readFileToUniversalInformation(filepath)
    #Read the file in the given location
    #Return an object containing the information set in the header
end

This is the other big TODO in moleculeReader. The top paragraph of an input file will be used to specify how the program should execute. It ought to be able to contain information like calculation type (RHF, UHF, DFT…), basis set to use (sto3G, p321, etc), specify the unit of measurement being used in input/output (angstroms, nanometers, bohr radii) (though note that internal calculations will all be done in nm. No sense making things needlessly hard), and probably other things I haven’t thought of yet. A custom type will be used to store this information accessibly.

moleculeDrawer.jl: module for creating 3D graphs of molecular structure
Sadly, Gadfly does not support 3D graphs yet. Since there’s not really any good way to visualise a molecular structure using only 2D graphs, we’ll just have to deal with using Plotly.

function plotMolecule(atomArray::Array)
    #create a graph displaying the atoms in the positions given by the array
    trc = scatter3d(
                    ;x=atomArray[:, 2], y=atomArray[:, 3], z=atomArray[:, 4],
                    mode="markers+text", opacity=0.5, marker_size=12,
                    text=map(x->getElementSymbol(x), atomArray[:, 1]),
                    textposition="center")
    #TODO make some of these things into inputs to this fn
    #TODO make different elements different colours :3yes
    layout = Layout(margin=attr(l=0, r=0, t=0, b=0))
    p = plot(trc, layout)
    return p
end

This is how you plot a 3D graph using Plotly. Important parts: x=atomArray[:, 2], y=atomArray[:, 3], z=atomArray[:, 4] gets the x, y and z columns of the array of atom positions. text=map(x->getElementSymbol(x), atomArray[:, 1]) creates the 4th column, the text of each atom. The text is just a label giving the elemental symbol.

function getElementSymbol(atomicNumber::Int32)
    #Retrieve the symbol for a given atomic number

    #The file is properly ordered, so the nth element is the nth line #EZPZ
    l = open(readlines, eledata)[atomicNumber]
    #Note that this loads the entire file into an array. May result in large
    #memory usage if the number of chemical elements increases,
    #or if using a table with very large amounts of info for each element
    elemInf = map(strip, split(l, ','))
    return elemInf[2]
end

Next TODO: create a better importer for the pt-data csv files. It should create a dictionary of Symbol:Number, and a simple list of symbols. That will make accessing symbols as we need them easier, and make it possible for moleculeReader to read symbols. Also find some similar files with a proper source, the source on these ones has vanished.

PlottingCodeTest
Fig. 1: Incredibly hard-to-make-out plot of atoms in non-meaningful positions. TODO: learn how to make this less awful to look at (in Atom? Which has a dark background? Or, learn how to make it display outside Atom, on a pale background? Or, make it export images without including the background?)

basis.jl: module for importing basis sets.
For the conversion from the PyQuante Python basis sets to the simpler, if more in need of interpretation, csv basis sets, see the python script basisReformatter.py. The reason for the conversion is to have a more language-neutral, but still somewhat readable version of the basis sets to hand.

function addLine!(b::BasisSet, line::String)
    l = map(strip, split(line, ','))
    linenum = parse(l[1])
    linearr = formArray(l[2:end])
    b.basis[linenum] = linearr
end

Note that the function!() format indicates a function that alters its arguments in-place rather than returning a new variable (though it is not a required syntax of doing so). This is the typical behaviour in Julia, so I’m not sure why it merits being highlighted. Opinions are divided over whether this is a good way of doing things; personally I quite like it.

function formArray(line::Vector)
    #In the input list, elements alternate between letters and strings of
    #numbers separated by spaces
    A = Array{Any}(2,0)
    for i = 1:2:length(line)
        #i indexes letters, i+1 indexes blocks of numbers
        a = line[i]
        b = listPairedNumbers(line[i+1])
        A = hcat(A, [a,b])
    end
    return A
end

The array we end up forming is not particularly easy to visualise – overall the structure is a hashmap relating numbers to (lists of (pairs of letters and (lists of pairs of numbers))). Uhh… Yeah. Think that’s right.

So far basis.jl is the part that probably needs least work. It does a single job quite effectively, and probably/hopefully doesn’t need any massive extension.

tester.jl: file for testing other modules.
Python is uncommon in having the wonderful if __name__ == "__main__": structure for testing modules by simply running them as main. In Julia, a more dedicated approach is required. So this file simply contains a set of functions for testing each other module.

try
    testBasis()
finally
    #This undoes the inclusions and uses, except when it doesn't
    workspace()
end

Annoyingly, the workspace() command doesn’t seem to do what it’s supposed to. To explain, consider the following development workflow in the Julia add-in for Atom, Juno:

  • Write a function in a module.
  • Add some code to the tester script’s function for the changed module to check the new function, execute the tester script.
  • Either part of the new code fails in some way.
  • Make further changes, run the tester code again.
  • Best case scenario: output overwhelmed by hundreds of warnings
  • Worst case scenario: console errors

Wait, what?

As I understand it, you’re Not Supposed to import things into the same workspace twice in Julia. In the good case (only overwriting my own code), the warnings just make things really hard to work with. In the bad case (Plotly), the workspace just dies. What the workspace() command should be doing is refreshing the workspace, replacing the main module with a new, empty one. In practice it doesn’t seem to prevent these problems. The only thing to do is to stop the Julia REPL and restart it.

JuliaWhy
Fig. 2: Second-run output interrupted by dozens of warnings. I have no idea how to prevent this “properly.” There doesn’t seem to be any way to suppress warnings.

Major improvements to make: Have input file set program behaviour. Allow element symbols in input file. Make graph output readable.

Next step: Add restricted Hartree-Fock algorithm. Add some test cases. Create some documentation on usage.

Celandine part 1 – Basics and Basis