twongkee's Blog

Unmodified firefox build (from this weekend) built on Debian on the VisionFive 2 !

Linux starfive 5.15.0-starfive #1 SMP Mon Feb 27 14:03:14 EST 2023 riscv64 GNU/Linux

not optimized, and not with the JIT, just the default build.

A few notes, if someone wants to replicate.

Using 202302 debian build. Enabled remote ssh.

• updated /etc/apt/sources.list to pull from regular location

#deb https://snapshot.debian.org/archive/debian-ports/20221225T084846Z unstable main
deb http://ftp.ports.debian.org/debian-ports/ unstable main

(this did “break” my X startup, so have to manually start Xorg from ssh session)

• installed rustc 1.67.1 via rustup
• installed node 19 from experimental tier
• had to build cairo from source, the one installed from apt gave error
  # error Cairo was not compiled with support for GObject

then Build Firefox on Linux

did have to install a lot of dependencies with apt.

further note: do have an nvme installed, and put the code on that, but left the root system on SD card. So very slow, took ~6 hours (with a few stops in between to add dependencies)

result: Firefox runs!

./firefox --version
Mozilla Firefox 112.0a1

Not very quick as the system is not optimized, and didn’t install the jit.

Youtube worked, and sound via headset (no hdmi sound).
a bit slow, so reduced resolution to lowest to get smooth video.
not bad for no GPU at all!

next project, to move root onto the nvme so system is faster. (and fix X again so boots properly)

arrived this week:

StarFive VisionFive 2 board

Picked up the 8 Gig dual network version from AmeriDroid.

Not had time to do much; Downloaded the 202302 debian image onto SD card. One hint: the board did not come configured to boot from SD: had to move the small switches to enable:

Once had figured that out, everything came up fine. Mine had no problems booting, and displaying on my HDMI monitor. Don’t intend initially to use it that way, so just installed ssh server, and unplugged. Now on my network and running headless (the little lights are flashing by the power). Am using my official Raspberry PI USB3 power supply, it has just the 3A power required.

uname -a
Linux starfive 5.15.0-starfive #1 SMP Mon Feb 27 14:03:14 EST 2023 riscv64 GNU/Linux

more updates to come!

sudo apt install pinhole

and “show all apps” (apparently camera is not mobile friendly yet)

click the camera app, and get the following pic!

left it sideways, since didn’t want to put up a “modified” picture.
the colours are also good.

Progress is moving quicker than I thought!

saw the update on mobian’s mastadon channel: here

Working, sort of?

This is my first time really looking at libcamera, I’m not a C person, so have not built much with C, so just trying out the basic build and utilities.

Downloaded the git repo and all the dependencies, and built on the phone.

above is another picture of the test image, on the pinebook pro using qcam. The colours are not modified by me, came directly from the qcam app. So better than megapixels out of the box.

But, the ui is not great, on my setup. The app screen image is very transparent, I can see the desktop image through the picture. Searching the web, seemed to indicate this may be a known issue? The lag is not bad, similar to megapixels. Now I have the pinephone keyboard connected, so I can type the filename into qcam interface without much difficulty, not sure how usable qcam is without it.

the cam cli seems to work also, if you make it capture 10 images, seemed to hang if you pass --capture=1 .


Did also seem to control the focus, managed to make some very blurry images, then some sort of clear ones. Will need to try to reproduce that; was switching between the older v4l-utils cli and libcamera, and think the settings bleed into each other.

next:
I’m more of a python person, so will try to build and play with the python extensions to get more control, and if that fails, try some (very bad) c code. Before anyone mentions them, I have downloaded the libcamera-apps, which should give more cli control, but they give me errors when trying to capture images, so likely need to be modified (or the mobian distro itself?). As mentioned I’m not a C expert, so will try the python stuff before digging into C/C++ directly.

note, mobian uname -a says:

Linux mobian 6.1-rockchip #1 SMP PREEMPT_DYNAMIC Fri Feb 3 05:58:59 UTC 2023 aarch64 GNU/Linux

Zooming in and looking at the triangle dot pattern, it is very regular. Changed the image type to png, which is non lossy, and see there are pairs of dots, in the regular pattern, on the green layer.

used GIMP to locate them

here we see the first pixel at 26,29.
the set of four are (in the box):
26,29
25,32
34,45
33,48

these then repeat, at 16 pixel intervals (both horizontal and vertical)

now check to see if is in the raw data file.

using numpy from python, can read the raw file, and using a jupyter notebook, as well as matplotlib and PIL, can display in the notebook the raw data (code pasted here, image of notebook below)

from PIL import Image
import numpy as np
from matplotlib import pyplot as plt
raw = np.fromfile('/home/mobian/Pictures/frame.raw', np.uint8)
# size
w=4208
h=3120
# read single dimensional raw image data array into square array
# starting at px, py, size square x square
px=0
py=0
square=60
plotarray=np.eye(square) # initialize square array
for y in range(0,square):
    for x in range(0,square):
        value = raw[w*(x + px) + (y + py)]
        plotarray[x][y] = int(value)
        
im = Image.fromarray(np.uint8(plotarray))
plt.imshow(im)
plt.rcParams['figure.figsize'] = (6,6)
plt.show()

and it also has the pairs of dots (screen shot of notebook below, with my specific frame.raw file)

now using the measurements, lets calculate the locations of all the dots, the 4 set of co-ordinates, repeated at 16 pixel (x) and 32 pixel (y) intervals

g=16 #gap
for x in range(0,w-4*g,g):
    for y in range(0,h-4*g,g*2):
        raw[26+x+(29+y)*w]= 255
        raw[25+x+(32+y)*w]= 255
        raw[34+x+(45+y)*w]= 255
        raw[33+x+(48+y)*w]= 255

and make them light. Rerun the same plot (slightly bigger and larger, but the same display code from above:

so now how to “remove” them?
simple test: replace with average of 4 “nearest” neighbours. Now due to how the colours are arranged, is not actually the pixel directly next to it, but 2 away (pasted code, but wraps, below screenshot)

g=16
for x in range(0,w-g*4,g):
    for y in range(0,h-g*4,g*2):
        #      y     x                  y-2                       y+2                             x-2                       x+2
        raw[w*(29+y)+26+x]= ( int(raw[w*(27+y)+26+x]) + int(raw[w*(31+y)+26+x]) + int(raw[w*(29+y)+24+x]) + int(raw[w*(29+y)+28+x]) ) /4
        raw[w*(32+y)+25+x]= ( int(raw[w*(30+y)+25+x]) + int(raw[w*(34+y)+25+x]) + int(raw[w*(32+y)+23+x]) + int(raw[w*(32+y)+27+x]) ) /4
        raw[w*(45+y)+34+x]= ( int(raw[w*(43+y)+34+x]) + int(raw[w*(47+y)+34+x]) + int(raw[w*(45+y)+32+x]) + int(raw[w*(45+y)+36+x]) ) /4
        raw[w*(48+y)+33+x]= ( int(raw[w*(46+y)+33+x]) + int(raw[w*(50+y)+33+x]) + int(raw[w*(48+y)+31+x]) + int(raw[w*(48+y)+35+x]) ) /4
raw.tofile('/home/mobian/Pictures/numpy.raw')  

result:

looks smooth!

the last line writes out to a new numpy.raw file.

which we then process with the same ffmpeg commands from the previous post:

original unprocessed / colour balanced + numpy fix

WordPress will let you zoom in to the full size images, if you want to look at the dots.

Still do not know where the dots come from, if someone with better imx258 docs, can send me a link?

I assume I need to set the sensor with different commands…

and still need to fix the focus

Could add these numpy and imagemagick commands to megapixels post processing…

p.s. all of the above (except GIMP) was run on the pinephone pro, including the jupyter notebook.