A Story About How We Fixed Vertica JDBC Driver for DBeaver

When connecting to a Vertica database through a DBeaver application a problem was detected (ce-4.2.2). If you enter a non-standard port, for example, 55555, the tool tries to connect to the default port 5433, anyway.

Screen 1

Diagnosis of the problems

Under a “Reason” line you can see that it failed to connect to the 5433 port, also there was a report that the client was going to connect to the 55555 port.

However, while entering a short number (1234), there is no such problem.

Screen 2

In this case, the connection failed on the specified port 1234.

Debriefing

Which port was an attempt to connect to, indeed? It is possible to run an IDA Pro tool and connect to a DBeaver process.

To convert Endianness, applications often use a set of functions “host to TCP/IP network” from a NetWare API. To convert a two-byte number (for example, a TCP port) it is possible to use “htons” functions.

“WinAPI” version of “htons” function is in “ws2_32.dll”. You need to find an import of “ws2_32.d” module and open a list of the functions.

Screen 3

Then, you need to set a breakpoint on “htons” function call.

Screen 4

If you set the 1234 port and click on “Test connection”, then debugging stops on the breakpoint.

Screen 5

First, you can notice that the function takes the value via stack.

00401A42 mov ebp, esp
00401A44 mov eax, [ebp + arg_0]

These instructions set the value into an EAX register from a top of the stack by offset to “arg_0”. “D2” and “04” bytes in the “Big Endian” format are our number 1234. But what if we indicate the number 55555 as a port? In theory, “03” and “D9” bytes should be placed on the top of the stack.

You need to click “Test Connection” again, but with 55555 port.

Screen 6

That is strange but here we have a number 5433 in the stack (15 and 39 bytes), which is a default port of DBMS “Vertica”. It can be assumed that the program sets a wrong port at once. We need to figure out where it happens. To communicate with DBMS Dbeaver uses JDBS and uploads a 6.1.2. version by default.

Screen 7

It’s possible to try decompiling a driver (.jar) and explore what happens before the connection. A “fernflower” software is good for this task. It supports a lot of options for decompiling of Java applications. For simple analysis a standard decompiling without flags is suitable.

Screen 8

Once completed, there will be decompiled source files.

You need to find the use of a default port “HPE Vertica” in a text editor, it might be able to understand the substance of the problem with the changing 55555 port into 5433.

Screen 9

There is an attempt to cast the port value to a Short Type in a “loadPreonnectSettings” method.

If it is not possible then the 5433 port will be used. A Short type in Java is limited by a range -32,768 – 32,767, therefore, the value 55555 will be an exception throw. It turns out, that the problem is in jdbc driver itself. Let’s try to fix its work.

Application patching

For “class” files patching it is possible to use DirtyJOE. You need to open “VConnection.class” from a jar archive. There is a list of this object’s methods in the “Methods” tab.

Screen 10

Double click on “loadPreConnectSettings” method opens CodeEditor.

Screen 11

A call of a getShort method occurs in 0000004E position, it means that B6 is an operation code of an instruction, and 003С is an operand label. It is necessary to change an instruction for getInt call for a Variant object.

Screen 12

There is a list of class methods in Constant Pool tab. A number 0060 is exactly the number of a method in a decimal.

Screen 13

A method getLnt was found below. You need to get back to a get Short call and change the command by getInt.

Screen 14

After that, you need to save the class and replace the original file in a jar archive. Before patching, Dbeaver connected to the 55555 port as to the 5433.

Screen 15

Now it is using the right port for connecting.

Conclusion

The problem as it turned out is not in the Dbeaver browser itself, but in the driver which it downloads for a communication with a Vertica database.

Update 09.19

Dbeaver removed the downloading JDBC in 4.3.0 release for Vertica. It means that the application will not use the problem version of this driver by default.