Docker with xHyve on Mac – access stopped container files

I had a problem with Docker on Mac OS X – nowadays it comes with xHyve VM instead of VirtualBox, and the VM uses cow2 format for it’s disk image, so it wasn’t clear how can one access files in /var/lib/docker on the VM.

Why does one even have to access the files? Well, there can be number of reasons. In my case it was Mosquitto container where Mosquitto MQTT broker is a PID 1 process, and when it’s not running – the container isn’t running either. So while changing a config file for Mosquitto I had to do some experimenting, which caused Mosquitto to fail on startup due to bad configuration.

As you can probably guess, it wasn’t possible to fix Mosquitto config otherwise than via /var/lib/docker, because I could not start the container anymore. So it was either this, or start from scratch with new container.

Anyhow, I did not find any working way to mount cow2 image, but I have found a solution on Docker forums to get terminal into xHyve VM when docker was started:

screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty

The username is root with no password.

Thanks to Rohinton Kazak for posting that answer.

S3 mock endpoint

This post is incomplete and more of a note for oneself – be advices.

Here’s the way I’ve been trying to mock S3 for local testing, using Ruby FakeS3 app: https://github.com/jubos/fake-s3

In order to make S3 endpoints point to localhost I had to create custom regions override XML and debug AWS Java SDK to figure out it’s format.

I also have to pass path of that file as system property in order for AWS Java SDK to use it:

-Dcom.amazonaws.regions.RegionUtils.fileOverride=/Users/user/.aws-regions-override.xml 
-Dcom.amazonaws.sdk.disableCertChecking=true 
-Dcom.amazonaws.regions.RegionUtils.disableRemote=true

I could not find a way to disable SSL (switch from HTTPS to HTTP) in Amason AWS SDK without modifying the code of apps (which I can’t do – otherwise I’d just set endpoint manually), thus I ran FakeS3 configured with some random certificates, and pass disableCertChecking property. Also RegionUtils.disableRemote property is included to reduce number of helper calls to actual AWS.

Still, I did not find any possibility to make AWS SDK use path style access – thus bucket name is prepended to my localhost:9000 URL configured in .aws-regions-override.xml file, and thus I still had to add some entries to /etc/hosts to make it point to localhost. But at least I did not break any real AWS URLs with that.

UPD: Also, in order for endpoint URI to change one has to call setRegion in code at least once. This works for Spring wrapper over AWS S3 SDK, but may not work for custom code, that instantiates S3 client without setting region.

The content of .aws-regions-override.xml is this (the “whatever” tag can have any name):

  
<whatever>
   <Region>
      <Name>localhost</Name>
      <Domain>localhost</Domain>
      <Endpoint>
         <ServiceName>s3</ServiceName>
         <Hostname>localhost:9000</Hostname>
         <Http>true</Http>
         <Https>true</Https>
      </Endpoint>
   </Region>
   <Region>
      <Name>us-east-1</Name>
      <Domain>localhost</Domain>
      <Endpoint>
         <ServiceName>s3</ServiceName>
         <Hostname>localhost:9000</Hostname>
         <Http>true</Http>
         <Https>true</Https>
      </Endpoint>
   </Region>
...
</whatever>

Paho MQTT client, max in-flight messages for QoS > 0

Working with MQTT protocol in Java usually means using Eclipse Paho FOSS library as a client (it’s even used by Spring for MQTT support in Spring Messaging).

Using Paho to send messages with Quality of Service (QoS) bigger than zero though might result in error/exception “Too many publishes in progress” in case many messages are sent in short period of time.

The straightforward fix to that is of-course not to use QoS other than zero, but there are other ways to remedy that problem.

Continue reading

ActiveMQ disable Diffie-Hellman ciphers to avoid “KeyUsage does not allow digital signatures” errors

Here’s how to do it:

transport.enabledCipherSuites=SSL_RSA_WITH_3DES_EDE_CBC_SHA

Add this parameter to URI in mqtt transportConnector (in your conf/activemq.xml config).

The need for this? I had a set of keys+certificates that were working perfectly fine on RabbitMQ, but on ActiveMQ I was getting “KeyUsage does not allow digital signatures” errors on client when it was validating server’s certificate.

I had no idea why this happened, googling revealed some fragmented info, in general I understood that my server’s certificate had “extension” “key usage” that indicated it didn’t allow (support?) digital signatures.

Screen Shot 2016-05-31 at 11.50.27 PM

Continue reading

Raspberry Pi Raspbian Jesse – free serial port ttyAMA0

I’ve posted the answer at stackexchange.

In short, disabling terminal on serial via raspi-config (advanced->Serial) should do the trick.

It it doesn’t for some reason – commenting out ttyAMA0 from /boot/cmdline.txt and disabling serial-getty via sudo systemctl mask serial-getty@ttyAMA0.service should definitely free the port.

But still one must manually set pins 15 and 16 into ALT0 state. Command-line “gpio” utility can be used for that:
gpio mode 15 ALT0
gpio mode 16 ALT0

UPD:
Don’t do the manual pin ALT0 mode setting – enable UART in /boot/config.txt instead (find enable_uart=0 and change to enable_uart=1).
This will ensure /dev/ttyAMA0 will exist. Otherwise it may not exist.

Move MP3s into folders by “Artist – Album – Year” in OS X

Quasi one-liner:

for filename in *.mp3; do; ART=$(mdls -name kMDItemAuthors -raw $filename | tr -d '\n"()' | awk '{$1=$1};1'); ALB=$(mdls -raw -name kMDItemAlbum $filename); YR=$(mdls -raw -name kMDItemRecordingYear $filename); echo $ART - $ALB - $YR - $filename; mkdir -p "$ART - $ALB - $YR"; mv $filename "./$ART - $ALB - $YR/"; done

Same, but on multiple lines:

for filename in *.mp3
do
  ART=$(mdls -name kMDItemAuthors -raw $filename | tr -d '\n"()' | awk '{$1=$1};1')
  ALB=$(mdls -raw -name kMDItemAlbum $filename)
  YR=$(mdls -raw -name kMDItemRecordingYear $filename)
  echo $ART - $ALB - $YR - $filename
  mkdir -p "$ART - $ALB - $YR"
  mv $filename "./$ART - $ALB - $YR/"
done

Spring Boot and Tomcat7

I had an issue with Spring Boot app @ Tomcat7 – the app was running fine at Tomcat8, but at Tomcat7 it wasn’t starting.

The app is using web.xml-less method – some class extends SpringBootServletInitializer and is annotated with @SpringBootApplication annotation. Again, this is sufficient at Tomcat8 for some reason, but not for Tomcat7.

Tomcat7 is Servlet 3.0, Spring Boot recent versions use Servler 3.1, as is Tomcat8, and this was part of the problem – because maven generated eclipse project was really willing to use Servlet 3.1 facet, which made it impossible to deploy to Tomcat7 with WTP.

In order to fix that, not only servlet version 3.0.1 but also Tomcat version 7.0.x had to be declared in pom. Then eclipse project had to be deleted (with all it’s dot meta files and folders) and recreated – this made it possible for me to at least deploy to Tomcat7 from Eclipse with WTP. Whew.

But – the app wasn’t starting. It was deployed, but Spring servlet didn’t register, no Spring contexts were initialized, etc.

It took me several hours to find a solution – and I’ve stumbled over it almost accidentally online. It turns out it’s insufficient for your @SpringBootApplication annotated class to just extend SpringBootServletInitializer – it should also explicitly implement WebApplicationInitializer! That’s it! This has fixed the issue – the app was running again.

I guess this is somehow required by the whole @HandlesTypes driven thing in servlet 3.0, but it is quite weird it works differently in servlet 3.1. Or is it just Tomcat? Well, apparently people are complaining about the same issue on… WebLogic.

Anyhow, the solution is found.

P.S. Just when I thought I’m done – I’ve run into same issue again, but on different environment – the problem moved from dev to test. Damn! I also had a bit older Tomcat on test, so checked with that version locally – worked fine. So took me a bit more to figure out switch from Java7 to Java8 is needed to fix the problem. Now I’m absolutely puzzled regarding the causes, but anyhow, the WebApplicationInitializer and Java8 made it work. Finally!