Where It All Started.

Where It All Started.

Life, Stock Trading, Investments, Business and Startup. Most are programming stuff.

Category: Development

Kintsugi Merge Testnet For Ethereum (ETH) Is Now Live

The Kintsugi testnet, the latest step in replacing Ethereum’s Proof-of-Work consensus method to Proof-of-Stake, has been deployed. The mainnet and beacon chains are expected to combine in Q1/Q2 of 2022. According to a release from ConsenSys, over 8.4 million ETH has been staked on Ethereum 2.0’s beacon chain.

Ethereum founding member Tim Beiko wrote in his announcement, “The Kintsugi testnet provides the community an opportunity to experiment with post-merge Ethereum and begin to identify any issues,”.

The Kintsugi testnet will help prepare for the “merge” to Ethereum’s 2.0. Following the merge, Ethereum 2.0 will move toward “Phase 2.” This will introduce sharding, a scalability feature that will improve fees and transaction times. Sharding is expected to arrive in late 2022.

Ethereum (ETH) Client Incentive Program To Reward Developers

Developers teams will get incentives in ETH that will be unlocked over time as part of the program. Each client team will receive 144 validators in total, or 4,608 ETH, worth about $17.8 million at current pricing. The program’s structure links teams with the network’s long-term health and guarantees that they are rewarded for developing secure software. The ETH 1 and ETH 2 Beacon chains will be merged sometime in the first quarter of 2022. Since the London update in early August, the network has burned 1.19 million ETH. At current pricing, this is worth over $4.6 billion, which corresponds to around 7,000 ETH (or $27 million) being burned per day.

The incentive program is setup to “ensure that client teams have a strong incentive to maintain the core Ethereum network over the long term.”. The teams eligible for the program include Besu, Erigon, Go Ethereum (Geth), Lighthouse, Lodestar, Nethermind, Nimbus, Prysm, and Teku.

The funds will be available immediately but withdrawals will be vested over several years.

Rebasing With Git

Rebasing is one of the features you probably want to have, if you plan to work on a neat git based project.


🍣 Where To Rebase?

If you know how many commits you make, to rebase you use git rebase with -i flag to enable interactive rebasing. The HEAD~<n> corresponds to the number of commits you have done (e.g. HEAD~4 if you have 4 commits to rollback to get to common ancestor commit).

git rebase -i HEAD~<n>

Sometimes, you commit a lot and forgot how many commits you’d make. To know the least common anscestor you have with master, you do git merge-base with your branch name as parameter.

git merge-base <your-branch> master

The above command will return a git hash which you can use on the git rebase command.

If you already know the git hash, then you can rollback to that specific commit and moving all current changes to unstaged. Once, the editor pop-ups you will choose which commit to retain, squash, and reword.

git rebase -i <git-ref-hash>

🍣 Merge Latest From Master

If you’ve already rebased your changes and needed to get lastest changes from master. All you have to do is rebase to the latest changes from master.
This command will do that.

git rebase origin/master

In any case, you’ve encountered some conflict first resolve it then continue in rebasing instead of creating new merge commit.

git rebase --continue

🍣 Overwriting Remote Repo Changes

Once all is done, overwrite your remote repo latest changes if you’ve pushed it. This will do a force push ignoring current ref on remote repo.

git push -f

🍣 Did Something Wrong? In Need Of Rollback

Did something wrong on merging conflicts? Don’t worry you can still see your previous changes using the command git reflog short for reference log.
You can checkout the reference hash then re-merge your changes.

git reflog

References

Limit Window Subsystem Linux v2 (WSL2) Resources To Speed Up Kubernetes

Window Subsystem Linux v2 (WSL2) is an iteration of the VM created by Microsoft, from Hyper-V to WSL and this the second generation of WSL. If it’s your first time accessing WSL2, it automatically provide you with the default setup which doesn’t provide any limits accessing your full workstation resources (CPU, RAM and other HDD). It means that if you have 8 cores cpu and 16Gb memory, it will use all that up. The problem with it is sometimes it affects your host computer and it gets slow. So to solve that problem we try to limit the resource consumption of WSL2.

Photo by Sadik Brika on Unsplash

Limit WSL Resource Consumption

On your profile directory %USERPROFILE% create a new file named .wslconfig. Set it’s content to the following:

[wsl2]
memory=8GB
processors=8

Change the settings base on your workstation capability, and this is what works for me.

Next, open up a powershell terminal in administrator mode and restart the LxssManager as this manages WSL2.

Get-Service LxssManager | Restart-Service

You could also use the wsl --shutdown method to restart WSL. Check if the vmmem process still consumes beyond its limit.

Troubleshoot

If the changes still not reflecting, try to restart your machine and also restart Docker Desktop.

Simple Rust Mutation Relationship Diagram

Rust mutation can be somewhat confusing if your a beginner. Its similar to C++ way of doing things on where to put the asterisk (*) and ampersand (&) sign in variable declaration. Moving the asterisk sign and ampersand sign makes the declaration sometimes more mutable and also can make it less mutable.

Here is a simple diagram on Rust mutation that I found on StackOverflow (SO). I can’t find the exact link to reference as this one is stored in my notes.


        a: &T == const T* const a;      // can't mutate either
    mut a: &T == const T* a;            // can't mutate what is pointed to
    a: &mut T == T* const a;            // can't mutate pointer
mut a: &mut T == T* a;                  // can mutate both

Converting Rust String To And From

Rust &str and String is different in a sense that str is static, owned and fix sized while String can be dynamically allocated once and be converted to mutable to be appended. Most of the time you’ll be working with String on Rust when re-allocating and moving values between structs.


There are times you may need to convert dynamic string to char bytes and static string. Here are ways to do it:

From &str

  • &str -> String has many equally valid methods: String::from(st), st.to_string(), st.to_owned().
    • But I suggest you stick with one of them within a single project. The major advantage of String::from is that you can use it as an argument to a map method. So instead of x.map(|s| String::from(s)) you can often use x.map(String::from).
  • &str -> &[u8] is done by st.as_bytes()
  • &str -> Vec<u8> is a combination of &str -> &[u8] -> Vec<u8>, i.e. st.as_bytes().to_vec() or st.as_bytes().to_owned()

From String

  • String -> &str should just be &s where coercion is available or s.as_str() where it is not.
  • String -> &[u8] is the same as &str -> &[u8]: s.as_bytes()
  • String -> Vec<u8> has a custom method: s.into_bytes()

From &[u8]

  • &[u8] -> Vec<u8> is done by u.to_owned() or u.to_vec(). They do the same thing, but to_vec has the slight advantage of being unambiguous about the type it returns.
  • &[u8] -> &str doesn’t actually exist, that would be &[u8] -> Result<&str, Error>, provided via str::from_utf8(u)
  • &[u8] -> String is the combination of &[u8] -> Result<&str, Error> -> Result<String, Error>

From Vec<u8>

  • Vec<u8> -> &[u8] should be just &v where coercion is available, or as_slice where it’s not.
  • Vec<u8> -> &str is the same as Vec<u8> -> &[u8] -> Result<&str, Error> i.e. str::from_utf8(&v)
  • Vec<u8> -> String doesn’t actually exist, that would be Vec<u8> -> Result<String, Error> via String::from_utf8(v)

Coercion is available whenever the target is not generic but explicitly typed as &str or &[u8], respectively. The Rustonomicon has a chapter on coercions with more details about coercion sites.


tl;dr

&str    -> String  | String::from(s) or s.to_string() or s.to_owned()
&str    -> &[u8]   | s.as_bytes()
&str    -> Vec<u8> | s.as_bytes().to_vec() or s.as_bytes().to_owned()
String  -> &str    | &s if possible* else s.as_str()
String  -> &[u8]   | s.as_bytes()
String  -> Vec<u8> | s.into_bytes()
&[u8]   -> &str    | s.to_vec() or s.to_owned()
&[u8]   -> String  | std::str::from_utf8(s).unwrap(), but don't**
&[u8]   -> Vec<u8> | String::from_utf8(s).unwrap(), but don't**
Vec<u8> -> &str    | &s if possible* else s.as_slice()
Vec<u8> -> String  | std::str::from_utf8(&s).unwrap(), but don't**
Vec<u8> -> &[u8]   | String::from_utf8(s).unwrap(), but don't**

* target should have explicit type (i.e., checker can't infer that)

** handle the error properly instead

Move Docker Desktop Data to Another Location (WSL 2)

In Docker Desktop for Windows the WSL2 version, you don’t usually have options to increase memory and diskspace as it will be managed directly by Windows.


The Docker Desktop data can be found originally in this location %USERPROFILE%\AppData\Local\Docker\wsl\data.

🚚 Export Docker Data

In order to make this work, first shutdown Docker Desktop. This can be done by right-clicking the system tray icon of Docker then from the context menu Quit Docker Destop.

Next is open your command prompt and type the following:

wsl --list -v

On which, when run will return to you the state of all WSL images.

  NAME                   STATE           VERSION
* docker-desktop         Stopped         2
  docker-desktop-data    Stopped         2

After that we export the docker-desktop-data into a tar archive. We will assume you are planning to move the docker data into D: drive, and within the drive you have already created a folder named Docker.

wsl --export docker-desktop-data "D:\docker-desktop-data.tar"

Next, is to unregister docker-desktop-data from WSL.
This command below will delete ext4.vhdx from %USERPROFILE%\AppData\Local\Docker\wsl\data\ext4.vhdx, so make sure you back it up first.

wsl --unregister docker-desktop-data

🚛 Import Docker Data

After export, we do import docker-desktop-data back to WSL.

wsl --import docker-desktop-data "D:\Docker" "D:\docker-desktop-data.tar" --version 2

The ext4.vhdx will now reside in the D:\Docker folder. Start Docker Desktop and verify the changes.

If everything works out, you can now delete the tar archive you created earlier D:\docker-desktop-data.tar. Please don’t delete the ext4.vhdx, otherwise you would lose all your images and containers in docker.

In case docker icon turns red in Docker Desktop, clear the docker cache which can be found in Docker Desktop settings.

Install Istio to Docker Desktop (WSL 2)

Istio is an open source service mesh. The documentation for installing it on windows is a bit vague, and the website only provides documentation for nix based systems.

Install Istio

Download the istioctl.exe executable from the official istio releases page and extract then place it in a folder your choice. Next will be adding it to your environment variable.

Adding the to environment variable will be straight-forward, add the directory location of the istioctl to PATH.

Install the istio namespace and services in kubernetes.

istioctl install --set profile=default -y

Also set istio to automatically inject Envoy sidecar proxies when deploying applications and services to default namespace.

kubectl label namespace default istio-injection=enabled

Troubleshoot

Determine if Kubernetes cluster is running in an environment that supports external load balancer.

kubectl get svc istio-ingressgateway -n istio-system

Check if there are any problems presented in analyze.

istioctl analyze

Also, check the endpoint if its empty or returning any headers and data.

curl -H curl -s -I -HHost:keycloak.7f000101.nip.io http://127.0.0.1

  1. Service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between services or microservices, using a proxy.
  2. Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation.
  3. Istio is a service mesh—a modernized service networking layer that provides a transparent and language-independent way to flexibly and easily automate application network functions.

Tunnel In Existing SSH Connection

Remote work, is a blessing and sometimes nightmare depending in your line of work. I’ve been in a situation where I’m connected to a remote workstation but due to some technicalities I’m not allowed to disconnect the current SSH1 connection and or create a new one. And where it lies, I need to tunnel a service from the remote workstation to my local machine.

So here’s how I did it!

The real voyage of discovery consists not in seeking new landscapes, but in having new eyes.

— Marcel Proust.

So where do we start?

Once you have an existing SSH session opened using the default OpenSSH2 client, to open a tunnel simply type <enter>~C where <enter> is the key on your computer keyboard.

~ (tilde) is the SSH’s default EscapeChar. You press <enter> first to clear the buffer, the ~ escape char and any one of a number of options.

If all goes well it will bring up a new console associated with your local SSH client, that will accept SSH command flags, which includes -R and -L.

To map a server service to your local workstation you need to use -L flag. The arguments for that flag would be [bind_address:]port:host:hostport but normally the bind_address is optional.

Then if you want to map local service and tunnel it to remote server, you’ll need to use -R flag. This flag holds similar arguments to the -L.

For example, if I want to forward a remote server Nginx deployed website and access it locally (with local bind IP). What could I do is type <enter>~C then -L 80:localhost:8080<enter>, after that I will immediately gain access to that when I access the site using localhost:8080 on my local machine.

To get a full list of escape sequence that the OpenSSH client accepts, type <enter>~?:

Supported escape sequences:
 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend ssh
 ~#   - list forwarded connections
 ~&   - background ssh (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice

That’s all guys. 🐲

Conclusion

Most of the command line tools have flags you probably haven’t explored. So try to explore each one to become proficient in the platform you are currently working on. Just like programming, you won’t memorize it on a day, but to truly know the tools capability you must use it in a very dire situation.

This OpenSSH2 escape sequence is really helpful for DevOps and software engineers (for software development).

Let me know in the comments if you have questions or queries, you can also DM me directly.

Follow me for similar article, tips, and tricks ❤.


  1. SSH or Secure Shell is a cryptographic network protocol for operating network services securely over an unsecured network. ↩︎
  2. OpenSSH (also known as OpenBSD Secure Shell) is a suite of secure networking utilities based on the Secure Shell (SSH) protocol, which provides a secure channel over an unsecured network in a client–server architecture. ↩︎

Dirty Logging With Serilog For ASP.NET 5

Here we are again on yet-another tutorial on using Serilog1 as your primary logging framework for both development as well as production.

If you look on the internet you’ll see there are many ways to integrate Serilog in your existing application, on this tutorial I’ll show you what I’ve been using for all the projects that I’ve handled.

You may ask what is the difference of my setup compared to others?


Programming isn’t about what you know; it’s about what you can figure out.

— Chris Pine.

The setup I’ll show you allows easy addition of new sinks and on the fly modification of log format without the need of recompiling your app again.

So how does that work?

Follow me and let’s dive on how to implement it. 👆

Prerequisites

First of all, you must have a .NET 5.0 SDK (Software Development Kit) installed in your computer and also I assumed you are currently running Windows 10 or Linux with proper environment set.

If you are on Windows 10 and already have a Visual Studio2 2019, just update it to the most recent version, that way would ensure your system to have the latest .NET Core SDK3 version.

So where do we start?

First, let’s create a test bed project. Type the command below on an existing shell console (e.g. bash, power shell, cmd, .etc).

dotnet new web -f net5.0 --no-https --name SerilogDemo

The command above will create a new project that will use .NET 5 framework. And from the above flags --no-https will setup the project to use non-secured (non-SSL) empty web API (it means will not generate a cert and an HTTPS URL).

After the project creation, change directory to the project root. And install the following nuget dependencies.

dotnet add package Serilog
dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Extensions.Hosting
dotnet add package Serilog.Extensions.Logging
dotnet add package Serilog.Settings.Configuration
dotnet add package Serilog.Sinks.Console
dotnet add package Serilog.Sinks.File

Then we will now dive to C# implementation, first import the Serilog library on Program.cs.

using Serilog;

After that, still on the Program.cs file go to the method CreateHostBuilder and add /or chain the UseSerilog method to our application startup.

webBuilder.UseStartup<Startup>().UseSerilog();

The UseSerilog call will initialize the Serilog instance. Then we move to Serilog hooks to our main thread. On the Main method convert it first from void to int, the reason for this is to be able to return different numeric exit code when an error happens.

Then we initialize the appsettings configuration early on, normally this will be initialized on startup. The reason we initialize this first, is we will use the appsettings to provide configuration to Serilog logger instance that will be called on the this Main method.

IConfigurationRoot configuration = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile(path: "appsettings.json", optional: false, reloadOnChange: true)
    .Build();

Next on, initialize the logger instance and handle further exceptions coming from the host builder creation. Will just use generic Exception to catch all types of exception (this is not advisable specially in production environment – if ever use specific type exception).

Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(configuration)
    .CreateLogger();

Log.Information("SerilogDemo Args: {a}", args);

try
{
    var host = CreateHostBuilder(args).Build();
    Log.Information("Starting serilog demo service");
    host.Run();
    return 0;
}
catch (Exception ex)
{
    Log.Fatal(ex, "SerilogDemo service terminated unexpectedly");
    return 1;
}
finally
{
    Log.CloseAndFlush();
}

When all changes done, re-check all your codes above. Check whether its the same setup, if all is fine we move to the appsettings.json as that is our main key in this tutorial.

using System;
using System.IO;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Serilog;

namespace SerilogDemo
{
    public sealed class Program
    {
        public static int Main(string[] args)
        {
            IConfigurationRoot configuration = new ConfigurationBuilder()
                .SetBasePath(Directory.GetCurrentDirectory())
                .AddJsonFile(path: "appsettings.json", optional: false, reloadOnChange: true)
                .Build();

            Log.Logger = new LoggerConfiguration()
                .ReadFrom.Configuration(configuration)
                .CreateLogger();

            Log.Information("Token Validator Args: {a}", args);

            try
            {
                var host = CreateHostBuilder(args).Build();
                Log.Information("Starting token validator service");
                host.Run();
                return 0;
            }
            catch (Exception ex)
            {
                Log.Fatal(ex, "Token validator service terminated unexpectedly");
                return 1;
            }
            finally
            {
                Log.CloseAndFlush();
            }
        }

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>()
                        .UseSerilog();
                });
    }
}

On the appsettings.json, add the following JSON structure inside the root.

"Serilog": {
  "Using": [
    "Serilog.Sinks.Console",
    "Serilog.Sinks.File"
  ],
  "Enrich": [
    "FromLogContext",
    "WithMachineName",
    "WithThreadId"
  ],
  "WriteTo": [
    {
      "Name": "Console",
      "Args": {
        "outputTemplate": "[{Timestamp:HH:mm:ss} {Level}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}{NewLine}"
      }
    },
    {
      "Name": "File",
      "Args": {
        "path": "logs\\log_.txt",
        "rollingInterval": "Day"
      }
    }
  ]
}

Analyze the JSON structure, if you look carefully on the Using its specified to use the Console, and File. What does this mean is it will log on console and the specified log file.

Then on Enrich key, its specify what type of data needed to be added on the log and where to get it. The WriteTo key allows you to configure each sink (e.g. change format of log).

After that, we will now check if everything’s working fine by compiling it and running the program. If everything is okay, you should now see logs in your console and a file will be created containing the previous logs.

dotnet run

That’s all basically, if you want to use the Serilog on your controller check below.

(OPTIONAL)

If you want to use Serilog on a controller or any class, then what you will do is import first the abstract logging class.

using Microsoft.Extensions.Logging;

Then on controller or class constructor add the ILogger with the name of the current class. Below is the sample basic setup.

private readonly ILogger<InformHub> _logger;

public InformHub(ILogger<InformHub> logger)
{
    _logger = logger;
}

public async Task Command()
{
    _logger.LogInformation("Hello, World!");
}

To use the logger on class, just call the logger instance and add call to the method of log verbosity you want. On the above we use LogInformation or INFO verbosity.

(ANOTHER OPTIONAL)

This is for IIS, if you ever want to use a file logging you may encounter that on default setup it will not log to files. In order to log to files, first create the log directory (specified on the appsettings.json). Then change the folder’s ACL settings, add the the user IIS_USRS\<name-of-iis-site-name>.

Fig. 1: Properties folder

Add Write access to the log folder for the user IIS_USRS\<name-of-iis-site-name>.

Fig. 2: Advance permission editing modal

Then just restart the site and check if everything is working. If its still not working just redo it base on above.

That’s all guys!

Conclusion

Logging is one of the most important thing to do in development and production. Having no loggers in your web application is not advisable (with the exception of fully tested, high security application), loggers can easily point out easily what exception and details occurred on client side.

You can found the complete repository here.

Let me know in the comments if you have questions or queries, you can also DM me directly. Follow me for similar article, tips, and tricks ❤.


  1. Serilog is a diagnostic logging library for .NET applications. It is easy to set up, has a clean API, and runs on all recent .NET platforms. While it’s useful even in the simplest applications, Serilog’s support for structured logging shines when instrumenting complex, distributed, and asynchronous applications and systems. ↩︎
  2. Microsoft Visual Studio is an integrated development environment (IDE) from Microsoft. It is used to develop computer programs, as well as websites, web apps, web services and mobile apps. Visual Studio uses Microsoft software development platforms such as Windows API, Windows Forms, Windows Presentation Foundation, Windows Store and Microsoft Silverlight. It can produce both native code and managed code. ↩︎
  3. .NET (previously named .NET Core) is a free and open-source, managed computer software framework for Windows, Linux, and macOS operating systems. It is a cross-platform successor to .NET Framework. The project is primarily developed by Microsoft employees by way of the .NET Foundation, and released under the MIT License. ↩︎