this was working as of 2020-05-23

apt update
apt install -y mosquitto mosquitto-clients
sudo systemctl enable mosquitto.service

Test the system to ensure it’s working. Start terminal windows…

term 1 – subscriber to the topic “test”

mosquitto_sub -t "test"

term 2 – publish something to the topic “test”

mosquitto_pub -m "message from mosquitto_pub client" -t "test"

Create a user/password

sudo mosquitto_passwd -c /etc/mosquitto/passwd username
Password: password

edit the file as root, /etc/mosquitto/conf.d/default.conf:

allow_anonymous false
password_file /etc/mosquitto/passwd

Restart

sudo systemctl restart mosquitto

test pub/sub in two terminals.

term-1 $> mosquitto_sub -t "test" -u "username" -P "password"
term-2 $> mosquitto_pub -t "test" -u "username" -P "password" -m "message from mosquitto_pub client"

That’s the setup of MQTT

Tasmota portion:

Tasmota is another beast… it wasn’t readily appearant to me when I was looking at the docs.

Tasmota 8.3.1

Enable MQTT support in tasmota:

1. Configuration -> Configure Other -> click MQTT Enable
2. Configuration -> Configuration MQTT
3. Host: the ip of the mqtt server
4. Port: should be 1833, unless you overrode it in the config.
5. Client leave as is.
6. User: the user you created for MQTT
7. Password: same password as you created earlier.
8. Topic: this should be unique for this device
9. Full Topic: %prefix%/%topic%/

Now, it wasn’t immediate appearant to me, what the mqtt topics where:

To turn on/off my device,

topic: "cmnd/aquriumlight/POWER"
payload: "on" / "off"

Groups are possible, the default group is “tasmotas”. The example below would turn on/off all devices listening to tasmotas.

topic: "cmnd/tasmotas/POWER"
payload: "on" / "off"

The device can be part of multiple groups:

issue the command in the console:

grouptopic
23:17:18 MQT: stat/aquariumlight/RESULT = {"GroupTopic1":"tasmotas","GroupTopic2":"","GroupTopic3":"","GroupTopic4":""}

GroupTopic1 is tasmotas, and topics 2-4 are empty.

to change that, in the console:

GroupTopic2 LivingRoom

sending a message to topic: “cmd/LivingRoom/Power” “off” would shut down all devices in LivingRoom.

to upgrade all devices (dangerous)

cmnd/tasmotas/upgrade

After you got the first tasmota devices setup, subsequent devices will be easy with mqtt.

MQTT Explorer

Using MQTT Explorer, it’s a GUI that will allow you easily send and view messages on the queue/topics. I recommend you check them out.

Connect to your mqtt server with the appropriate credentials.

Try out a few things:

EXMAPLE: Set your timezone on all devices to eastern daylight savings time. (GMT-4)
TOPIC: cmnd/tasmotas/timezone
PAYLOAD: -4

EXAMPLE: Set your timezone, and location on all devices to pacific, and location is Googleplex. (GMT-7)
TOPIC: cmnd/tasmotas/Backlog
PAYLOAD: timezone -7; latitude 37.422; longitude -122.084

I recently swapped servers, as my old one was running Ubuntu 14.04, I decided to just swap out the server completely to Ubuntu 20.04, port the data… hence the recent rsync post.

The process was simple:

• create new server
  • setup apache (vhosts), mysql, php, etc…
  • firewall rules should be the same, so you could use the same security group. A review should be done.
• rsync data over
• export database
• I did a smoke test, by editing my hosts file to point to the new server.
  • install software
  • restore data/backups.
• unbind/detach IP from old server
• repeat for new server.
• bind/attach old IP to new server
• Setup SSL certificates if you haven’t already.

Now, my old server, didn’t allow for IPv6… the new one did.

Create a new AAAA record, @/lloydleung.com to the IPv6 address.

I did have a mistake somewhere, where I created an “A” record for the www.lloydleung.com. I changed that to a C record, where www points to @.

Now when something tries to go to “www.lloydleung.com”, either IPv4 or IPv^, it looks up “lloydleung.com”, with the corresponding A or AAAA record.

Set your TTLs to something low, until you know it won’t change.

Many people now have Amazon echo/dots / Google home / Apple HomePod, smart TVs, smart thermostats, etc…

What happens if one of those get hacked? They’re already on your system? they’re authorized and allowed on your network… now what?

That’s why you should have a WiFi network dedicated to IoT devices on a separate VLAN. If one gets hacked, they’re not on your network.

You allow your main lan, to cross the VLAN without issue, but devices on the other side, no nothing outside their VLAN and their connections to the net (which some need to get updates, weather, etc).

Things that should be on your IoT lan: alexa/google/siri devices. your smart TV/android players/apple TVs/PS4/PS3/Xbox/etc… including your smart locks, and smart thermostats. Your media server should also be on that network.

Things to keep on your secured main:
your smartphone
laptops/computers
servers (NFS/backups)

Your cameras that don’t need ANY connection to the net could also be firewalled up on another VLAN, or as some others have done, put in firewall rules for those devices to not be allowed to connect anywhere, but allowed to be connected to from inside the lan (network security camera recorders).

Some common firewall rules…

your main LAN to IoT:
* all traffic

your IoT to Lan
* mDNS — this is so the devices can be discovered by the entire network.
* your devices that need connection back across the VLAN. Such as a network scanner, to sftp into a server to drop scanned images off
* mqtt traffic, because that’s what a lot of IoT devices do use for communications.
* ntp traffic out to the net.
* you could probably block out most other traffic… if you know you can.

Your IP Cameras should probably be firewalled to prevent anything going out, with the exception of “All Established and Related connections”.

It’s easiest to show what “All Established and Related connections” means. Say you’re devices behind your current router make a connection to a website, that website is able to send you back data — but that server couldn’t originally start/initiate communication with your browser.

This is not a simple, easy task to make and split your network. It will take several hours, and hardware that is compatible with it. My network ecosystem is 100% Ubiquiti. I did have a dlink smart switch, that stripped off my vlan, so non of the devices on the vlan could communicate out. After figuring that out, I moved all the devices off, and will retire that switch.

I think maybe still possible to use a switched port, that is dedicated to a VLAN, so all devices behind it will be on that VLAN… I’ll have to test that, but I’ve had enough networking stuff for the last few days.

TL;DR;

rsync -aHAXxv –numeric-ids –delete –info=progress2 -e “ssh -T -c [email protected] -o Compression=no -x” <source> <target>

options:

a – archive
H – preserve hard-links
A – preserve ACLS
X – preserve extend attributes
x – don’t cross filesystem boundaries
v – verbose
–numeric-ids – don’t map uid/gid values by user/group name
–delete – delete extraneous files from destination dirs
–info=progress2 — show progression
-e “ssh -T -c [email protected] -o Compression=no -x”

<source>
<target>

The flags made the sync much faster…

arcfour is dead, for a long time… same with blowfish… so [email protected] is the next best thing.

It’s been a while since I posted anything… Kids/family have taken up a lot of time.

Dylan’s started to ride two wheels, learning about the Pedrin method was a little too easy to teach him.

Dylan was in training wheels for a while and was ready, as he could ride a few meters without either wheel on the ground.

Every child is difference, your child is ready when they feel ready, so there’s no point in pushing it. Dylan (age 5.75), was ready but scared about falling. To put that at ease, it’s pretty simple.

You as the parent, stand at the front of the build, with the front wheel between your legs, to prevent the bike from falling.

Ask the child to balance, they won’t be able to, the point them experiencing how it feels before falling to either side. Give them a few tries, encourage them either way. The point is so they know what to do by putting their feet down.

Now… just give them a push, and away they go… my son was able to go a meter or two the first time, but then a house length, then two… he still needs practice, but this spring has been crazy with the weather dropping to 6°C from 20°C shorts weather.

[update: 2020-05-20] There does seem to be a conflict with Let’s Encrypt certbot. You’ll need to manually run updates. I’ll update once I figure it out, how to do it more automated. Looks like possible race condition for the ghost, and then the SSL… but since you’re defining them by macro’s it’s not so bad.

steps seem to be:

1. DNS to point to the server
2. Create vhost only (not SSL yet), and ensure it works.
3. Certbot certificate only
4. Add Use VhostSSL $domain line.
5. Renewals are certificates only…? <– untested

[Original Post] I have had to manage my share of vhosts, and it’s the same vhosts all the time. Maybe slight changes. They usually look something like this:

<VirtualHost *:80>
    ServerName domain.tld
    ServerAlias www.domain.tld

    DocumentRoot /var/www/project/public
    <Directory /var/www/project/public>
        AllowOverride All
        Order Allow,Deny
        Allow from All
    </Directory>

    ErrorLog /var/log/apache2/project_error.log
    CustomLog /var/log/apache2/project_access.log combined
</VirtualHost>

But repeated for EVERY domain name on the server. There’s an easier way, since apache 2.4.5.

You define the template macro:

<Macro VhostMacro $domain>
<VirtualHost *:80>
    ServerName $domain
    ServerAlias www.$domain

    DocumentRoot /var/www/$domain/public
    <Directory /var/www/$domain/public>
        AllowOverride All
        Order Allow,Deny
        Allow from All
    </Directory>

    ErrorLog /var/log/apache2/$domain_error.log
    CustomLog /var/log/apache2/$domain_access.log combined
</VirtualHost>
</Macro>

You call the macro:

Use VhostMacro lloydleung.com
Use VhostMacro site2.tld
Use VhostMacro something.com

You can extrapolate as much as you want.

Remember to restart/reload apache to load the new configs. Otherwise apache won’t know.

Having trying to debug without xdebug felt like my hands were being tied behind my back.
Took a while, but my friend Bart was able to help me get this up and running, after lots of trouble shooting.
The way it seems, api-platform sets docker up in distinct containers:
client
db
admin
php
api
cache-proxy
proxy
Wanting to debug in xdebug, on a mac, inside a docker container wasn’t as straight forward as I would have thought it could be.
Docker/alpine is running nginx. Nginx proxies the calls for php-fpm.
The meat and potatoes of getting this setup:
file: api/Dockerfile

	pecl install \
		xdebug-${XDEBUG_VERSION} \
	; \
	docker-php-ext-enable \
		apcu \
		opcache \
		xdebug \
	; \

file: api/docker/php/php.ini

xdebug.remote_port=9000
xdebug.remote_enable=1
xdebug.remote_connect_back=0
xdebug.idekey = "PHPSTORM"
xdebug.remote_host = "10.254.254.254"

on your host machine run this on the command line:

$> sudo ifconfig lo0 alias 10.254.254.254

file: api/.env

PHP_IDE_CONFIG=serverName=docker

Run/Debug Configuration:


That should do it… change PHPSTORM, to your ide key… but PHPSTORM is pretty standard for PhpStorm.
Remember to rebuild your container… as sometimes (why I don’t know) changes don’t take effect until it’s rebuilt.
If you’re having troubles, look at “phpinfo();” or “$> php -v”. Check:

1. xdebug is enabled as a plugin
2. environment variable: PHP_IDE_CONFIG, is indeed set… and without quotes.
3. your absolute path on the server is correct

Hoped this helps others… as it wasn’t straight forward at all for me.
see: https://blog.philipphauer.de/debug-php-docker-container-idea-phpstorm/
todo for this post: fix links for dropbox images.

I was looking around, and couldn’t find anything.
 
Lots of people on home-depot felt that these things were not the greatest.
Problems that people ran into, where reusing the old tubes.  These are meant for a complete system.  It’s color coded for ease.
I was using the 1/4″ tubing so that it’ll fit my under the sink reverse osmosis filtration system.  I needed to connect the tank line to the fridge/icemaker.
I went and bought a T, and another coupler.
I could not find any instructions directly for the watts Ander-Lign compression fittings/coupler.  This is what I found to work (with no leaks):

1. Do not disassemble the unit.
2. Finger tighten the assembly
3. Cut the plastic tube flush
4. Insert the tube directly into the brass fitting, while the unit is STILL assembled.  Push until it doesn’t go any further.
5. Using a pair of wrenches, do a 3/4 (three quarter) turn of the nut to compress the brass sleeve.

Items purchased from home depot:

• Watts (A-12) Ander-Lign Compression Tee
  • For splicing off the reservoir tank
• Watts (A-4) Ander-Lign Compression Nut
  • For fridge connection
• Watts Polyethylene Tubing (SPEB25) 1/4″ x 0.170″ x 25 ft

Hope this is helpful for others, as I couldn’t find how much to tighten the nuts.  Seems to work for me… your milage may vary.

Objective: Allow Ubuntu to start BTSync on reboot.
 

Server

1. create an EC2 instance, with appropriate storage space.
   • Ubuntu LTS 64bit (currently 14.04)
2. download the BTSync client
   • http://www.bittorrent.com/sync The 64bit linux version.
3. dump the btsync.conf
   1. ./btsync –dump-sample-config > btsync.conf
4. edit the btsyn.conf for your settings.
5. sudo su
6. create /etc/init.d/btsync file
   • See below
7. chmod 755 /etc/init.d/btsync
8. update-rc.d btsync defaults
9. service btsync start
10. test the connection against other boxes.

Web GUI

1. Since we’re on EC2, and security is paramount.
2. Establish a port tunnel
   1. sudo ssh -i ~/.ssh/EC2KEY.pem -L 8888:127.0.0.1:8888 ubuntu@serverHostNameOrIP
3. Connect to the system via: http://127.0.0.1:8888/gui
4. Verify that all is working on the server end.

 

Client(s)

1. Connect w/the shared keys.
2. Verify that the system is working.
3. Put a file into the shared folder. Notice that the file should go up to the server.

 

On the Server EC2 instance.

create the file /etc/init.d/btsync
 

#!/bin/bash
# /etc/init.d/btsync
# @see: http://askubuntu.com/a/352240 source
if (( $EUID != 0 )); then
    echo "Please run as root"
    exit
fi
# Carry out specific functions when asked to by the system
case "$1" in
start)
    /USERACCOUNT/.btsync/btsync --config /USERACCOUNT/.btsync/btsync.conf
    ;;
stop)
    killall btsync
    ;;
*)
    echo "Usage: /etc/init.d/btsync {start|stop}"
    exit 1
    ;;
esac
exit 0

 
Update 2014-08-23

• changed the init.d script to bash, and added root check.
• Noticed that BTsync does not work well over a VPN connection.

Things of importance

1. Family photos/videos
2. Documents
   1. scanned docs
   2. license/registration information for software
3. Everything else.

 
 

How to backup, rule of 3-2-1

3 – (three) copies of everything of importance.

2 – (two) media types at least.  Remember floppies? Remember Zip disks? Beta tapes… yeah…

1 – (one) off site copy.  Home burns down… you’re fine, data wise… and insurance for the rest.

So, how do I implement this:

• BTsync/dropbox/box.com, to get offsite backups running quick and easy.  Sparkle share was a little too much effort, needing a git repo.
• BT Sync, will make multiple copies easily across your network, with super ease.
• my amazon ec2 instance will be a offsite node, which can then push to glacier or s3.

Eventually, I’ll write some crons to do the auto pushing to S3/glacier.  I’ll share them once I do (2014-08-03)