Saturday, April 23, 2016

Odroid + Minecraft = Possible

Introduction

ARM based CPU's have always interested me. Long before they had gained the reputation that they have today, I was hacking away on them. Finding out everything that I could do with them. Over the past few years, there has been a huge interest in optimizations for the architexture along with many operating systems being specificially(at least originally) developed for them. While I'm still working my way to replacing all of the hardware in my house with an ARM CPU, I feel at least in the server department, it should be an easy venture.

This has proven, however to be a bit more work than a few apt-get installs and we're off to the races. No, it's been a bit more than that, but, not much! Which I think is awesome!

The first major migration for me was to move a minecraft server over to an ODROID - XU4.

This little beast of a machine has proven to be quite the workhorse, and I may even try to expand on it even more with a webserver running and possibly file storage down the road. It is hosted at a friends house, as it's his, but I did the majority of the software work to migrate everything over.

A few specs on the XU4:

  • Samsung Exynos 5422
    • 4 A15 Cores @ 2Ghz
    • 4 A7 Cores @ 1.5Ghz
  • Mali-T628 MP6 GPU
  • 2 GB LPDDR3 RAM
  • Gigabit Ethernet
  • 2 USB3 Ports
  • 1 USB2 Port
  • 1 HDMI Port

Now, on paper, if you ignore the part that makes these cores ARM cores, this is a beastly machine. However, even though these are ARM cores, and not traditional x86(64) cores, they are still quite quick! From the get-go my only concern with the hardware was the 2GB of DDR3 RAM. This is where I had to work a bit of magic and modify the setup just a bit, to have a usable server for the amount of users that are normally present.

Now, onto the how!

Setup - How do I do that?

The XU4 has a few different options when it comes to storage, and I do not believe this is within the scope of this post, but for the most part, I would count on needing at least 8GB of storage for the software along with the actual map files needed. This is honestly on the small side though, so if you're thinking you're going to have a big world, I'd go for at least 16GB of storage.

Storage

The type of storage doesn't really matter. While an eMMC module would be optimal, using a MicroSD card or even a spinning disk drive with the cloud box for the XU4 wouldn't change that much. Minecraft does have a decent amount of disk activity, but not to the point where that will be your bottleneck. This server has a 16GB eMMC module which has been more than enough so far.

Software Setup

While the OpenJDK is available for the XU4, I would recommend the Oracle build of Java 8. I'm using the latest build as of writing this post. I did test the OpenJDK initially, and it worked just fine, but the Oracle Java package is more optimized, so out with the FOSS and in with Oracle. (I hope to not have to write that again)

Spigot's optimizations have proven to be the saving grace of running a Minecraft Server on the XU4. If you haven't heard of Spigot, it is a rewire to the popular Bukkit server, that focuses on optimizations for the server. Another main feature is the plugin API. While Spigot has (pretty much) full support for Bukkit plugins, it has, as far as I know, rewritten the API hooks to be even more optimal for, you guessed it, performance.

Which plugins should you be using?

While most plugins should be fine for using with the XU4, I've found there are a few quintessential ones that are really needed for optimal performance.

Plugin List:

These are the only plugins currently in use on the server, and I feel they are all really needed for a clean Spigot setup and also an optimal one. The key plugin in that list though is NoSpawnChunks. This plugin unloads the spawn chunks as if they were any other chunk in the gaame. This reduces the memory usage of the server by around 10% when people are online and in the spawn chunks because not all of them are loaded. Around 15% when they are online and not in the spawn chunks.

The downsides to this plugin, however, is that no processing is done in the spawn chunks when no one is online. What this boils down to is that farms that need to be built in the spawn chunks will not work, including Iron farms. Now, some versions of these farms will work if they do not require the farm to be loaded all the time, but anything that depends on the farm being loaded all the time will not.

I find this to be a minor setback as while a server iron farm is a nice thing to have, we are using it more as a restriction to force us to mine.

Some testing was completed without this plugin, and things are definetely still playable, even smooth, the problem being it further limits the number of players that can log on to the server. The decision was made more players was a better trade off.

Essentials and Essentials spawn are Bukkit plugins that are great for admining the server and allowing players to set warps. While far from required for this setup, I would recommend using them, just for ease of use.

To pre-generate or not to pre-generate

In some initial testing, terrain generation was proven to be the bottleneck of the XU4. While it was more than capable of generating terrain, it could not keep up with more than one person generating chunks at the same time. To resolve this issue, the decision was made to pre-generate the world. The map files were moved to a more powerful machineto speed up the process, but this was far from necessary. The generation was completed in about an hour with a modified server that moves the spawn point around and uses normal generation.

The downsides to being pre-generated means if there is an update that chagnes generation, players will need to move out quite a ways in the world to find this new generation, but again, this was a tradoff that was decided on deemed worth it by the player base.

The program used to pregenerate was MinecraftLandGenerator. While it eneded a bit of modification to work with our chosen seed and teh current version of minecraft, generation was completed in about an hour for a 10,000 by 10,000 area. This only generated the overworld, and not the nether or the end.

Research is being conducted on how to generate those regions, as they are now much more important in the current patch. The server hasn't progressed to the point of being in need of those regions yet though, so I have some time.

Optimization - What you need to know

In regards to optimization, there wasn't a whole lot that was done. The server OS in use is Ubuntu 14.04 LTS Minimal, which is available from the ODROID downloads page here. Basic scripts were put in place to start the server up in a tmux session. Currently there is 1.5GB of RAM dedicated to the Java VM and this is proving to be a bit overkill for the current playerbase, but being as there is no use for the rest of the resources on the server at the moment, the settings have stayed.

One thing that is currently being worked on is underclocking the GPU to save a bit on power, as this is a headless server. In the process a custom kernel may be compiled to help with performance as well, but it is far from needed. If you have any questions, please feel free to comment below or at the forum post started by the hoster on the ODroid forums.

Thursday, March 24, 2016

Moving To The Cloud


THE CLOUD


My Story

An adventure. That's the best way that I can describe this process. My development work, at least in this part of my life, has always been a side project, but I'd taken it somewhat seriously. I found myself, though, wandering off to do other things instead of concentrating on my true passion. Writing software. I hadn't really thought about solving the concentration problem because, who cares right? It's just a hobby, no need to get serious about it. I do have a degree in software engineering, but I'm not using it in my current position, so why bother?

The why bother? question is what really got me thinking though. That's not the self that I knew. That is someone totally different. I have never been the one to just settle. Be happy with what I know and move on. I have to keep learning, it's in fact always been one of my life goals. I don't want to stop learning. So, I started thinking of ways to solve the concentration problem.

Phase One

Phase one of solving the problem was a complete and utter failure, but that's okay. An attempt was made. I decided to remove distractions with pure and utter will power. Now, for someone with an addictive personality, I find, at least for myself, that just will power is not enough to force a change in my life. It has to be something drastic, unsettling, but, I tried anyway. Eventually I slipped back into wandering off to different things, gaming, media consumption, Internet wars, etc.. This was when I realized that I needed to do things a bit differently.

Phase Two

Phase two was when I started thinking of different ways to remove the temptation of distractions from my workflow if one could even call it that. I started looking into a laptop computer. Something if you'd asked me a couple years ago if I'd ever go back to, I would have called you crazy. I loved my desktop, and had built it with my own two hands to do exactly what I wanted it to do, but, alas, I needed to make a change. So, for a few months I researched into many different options. I eventually had made my mind up and decided to go back to a Mac. I started moving my workflow over to an OS X VM to see if I could quickly adapt to the OS again. I had no trouble and was quickly back into the groove. The problem was, the groove was still the distractions and unfocused groove that I was trying to avoid. I didn't end up purchasing a Mac, although, let me tell you, I was close. Very close. And I'm so happy that I didn't.

Phase Three

Enter Google Compute Engine(aka THE CLOUD). This is where I am now. I write this post in Markdown inside of VIM, and quite frankly I'm loving it. The distraction has gone away as I sit here, SSH'd into a GCE instance with very limited resources, but it's just enough to run all of my development tools. I did, however, have to make a change to my development style. A drastic one at that. Most of the tools I used on a daily basis already ran just fine on Linux as that is where I was doing my development before. What they didn't do, though, is run without a graphical user interface. They were (mostly) capable of doing so, but it was not something that I was used to. So, onto learning I did, and I still am. I have written a few Python scripts that wrap up some of the build tools that I was used to just clicking a button for, but that's okay. I'm learning, and that's been my whole goal.

Where to from here?

Well, Phase Three does address part of the concern about reducing distractions, but I want to take it a few steps further. I'm in the process of selling my beloved desktop computer that I built, and moving onto something a bit less powerful, but free of many of the distractions that I had before.

Google Pixel C - With Attached Keyboard
That will be a tablet with attached keyboard. Now, this is a huge jump from the three monitors I'm sitting in front of at the moment and multiple cores of x86 power backing them, but I think it's the right move for me. A move I should have made a long time ago in my life. A move that I hope will bring about more learning and fewer distractions. I'm looking into a couple of tablets at the moment, but I'm mainly focusing on the Pixel C by Google along with it's available keyboard accessory. My only worry is that when I want to take it mobile, the keyboard is lacking some keys that I would normally use on I'd say a semi-regular basis.

I don't know if that will be the tablet that I end up going with, but I do know, that if I want to keep learning and remove distractions from my workflow I have to change something. This desktop will not do. I hope to, a few months from now, come back and revisit this topic. As it's something I've become very passionate about. I do not think this is for every, or for every developers style. What I do believe, however, is that if you're having trouble with distractions, this would be a great direction to at least experiment with.

I have a few more posts in the works that have more detail about what tools I'm using, how I'm using them, and what I'm working on developing to help improve my workflow.

Inspirations

I thought I should mention what inspired me to make the move to a cloud based development approach, along with ideas on what to change my hardware out for. Links below.
There are more, but I seem to have lost the links to them. Irregardless, these posts might be a few years old, but things have only gotten better, not worse. Tablets have improved, cloud computing instances have gotten cheaper, it's just all around better. I will say that taking this route, you will most likely need to write a bit of custom toolery to get started, as not much exists for development based around the CLI anymore, at least not for the typical Joe Computer Programmer.

More to come, stay tuned!