Skip to content

Arrow feather deserialization of dataframe with column group #1044

Open
@ian-k

Description

When serializing dataframe with GroupColumn(s) after de-serialization the columns are represented as ValueColumns.

@test
fun testMultilevel() {
val rangeRows = (1..3).toList()
val downColsRange = 1..4
val middleColRange = 1..3
val topColRange=1..2

    val IID = columnOf(*rangeRows.toTypedArray()).map { "IID$it" }.named("IID")

    val topCols = topColRange.map { tc->
        columnOf( middleColRange.map { mc ->
            columnOf( downColsRange.map { dc ->
                columnOf(*rangeRows.toTypedArray()).named("$tc-$mc-down$dc").map{Random.nextDouble()} }
            ).named("$tc-middle$mc")
        }).named("top$tc")
    }

    val col2add = columnOf(*rangeRows.toTypedArray()).map { "NEW$it" }.named("Calc")

    val frame = dataFrameOf(IID,*topCols.toTypedArray())


    frame.writeArrowFeather(File("output2.arrow"))

    val arr = AnyFrame.readArrowFeather(File("output2.arrow"))
    println(arr.columnTypes())
}

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions