### Author Topic: Artificial neural network  (Read 214 times)

#### dclaudiu

• Newbie
• • Posts: 1 ##### Artificial neural network
« on: April 09, 2018, 03:15:49 pm »
I've made an Artificial neural network  for mnist. It dosen't have a bias and it gets 96.48 accuracy on the 10000 test data after 30 epochs, the optimum learning rate 0.28125 for first weights 1.546875 (0.28125*5.5) for the second and it takes 6 hours to train on my old laptop. If any one can make it cleaner, faster or more accurate then be my guest and modify it.
Code: [Select]
`DIM weight(23520), weight2(300), neuron(30), neurout(10), sum(30), sum2(10), neuroin(784) AS DOUBLE'generating random weightsFOR c = 1 TO 30    FOR a = 1 TO 784        pm = CINT(RND)        IF pm = 0 THEN sign = -1 ELSE sign = 1        weight(a + p) = RND * sign    NEXT a    p = p + 784    FOR d = 1 TO 10        pm = CINT(RND)        IF pm = 0 THEN sign = -1 ELSE sign = 1        weight2(d + q) = RND * sign    NEXT d    q = q + 10NEXT c'starting the trainingFOR epoch = 1 TO 30    OPEN "mnist" FOR BINARY AS #1    FOR s = 1 TO 60000        p = 0        q = 0        e = 0        f = 0        l = 0        o = 0        FOR r = 1 TO 10            sum2(r) = 0            sum(r) = 0            sum(r + 10) = 0            sum(r + 20) = 0        NEXT r        a\$ = INPUT\$(785, 1)        result\$ = MID\$(a\$, 1, 1)        result = VAL(result\$)        FOR t = 1 TO 784            b\$ = MID\$(a\$, t + 1, 1)            b = ASC(b\$)            neuroin(t) = (1 / 2048 * (b + 1))        NEXT t        FOR c = 1 TO 30            FOR a = 1 TO 784                b\$ = MID\$(a\$, a + 1, 1)                b = ASC(b\$)                sum(c) = sum(c) + ((1 / 2048 * (b + 1)) * weight(a + e))            NEXT a            neuron(c) = 1 / (1 + 2.71828 ^ (-(sum(c) / 1)))            e = e + 784            FOR d = 1 TO 10                sum2(d) = sum2(d) + (neuron(c) * weight2(d + f))            NEXT d            f = f + 10        NEXT c        FOR g = 1 TO 10            neurout(g) = 1 / (1 + 2.71828 ^ (-(sum2(g) / 1)))        NEXT g        DIM target(10), mrofer(10), doutsum(10), oldweight(300) AS DOUBLE        target(result + 1) = 1        FOR h = 1 TO 10            mrofer(h) = target(h) - neurout(h)            doutsum(h) = (neurout(h) * (1 - neurout(h))) * mrofer(h)            FOR i = 1 TO 30                oldweight(h + j) = weight2(h + j)                weight2(h + j) = weight2(h + j) + (0.28125 * doutsum(h) * neuron(i))                j = j + 10            NEXT i            j = 0        NEXT h        DIM errofhu(30) AS DOUBLE        FOR k = 1 TO 30            errofhu(k) = neuron(k) * (1 - neuron(k)) * (doutsum(1) * oldweight(1 + l) + doutsum(2) * oldweight(2 + l) + doutsum(3) * oldweight(3 + l) + doutsum(4) * oldweight(4 + l) + doutsum(5) * oldweight(5 + l) + doutsum(6) * oldweight(6 + l) + doutsum(7) * oldweight(7 + l) + doutsum(8) * oldweight(8 + l) + doutsum(9) * oldweight(9 + l) + doutsum(10) * oldweight(10 + l))            l = l + 10        NEXT k        FOR n = 1 TO 30            FOR m = 1 TO 784                weight(m + o) = weight(m + o) + (1.546875 * errofhu(n) * neuroin(m))            NEXT m            o = o + 784        NEXT n        target(result + 1) = 0        e = 0        f = 0        FOR r = 1 TO 10            sum2(r) = 0            sum(r) = 0            sum(r + 10) = 0            sum(r + 20) = 0        NEXT r        FOR c = 1 TO 30            FOR a = 1 TO 784                b\$ = MID\$(a\$, a + 1, 1)                b = ASC(b\$)                sum(c) = sum(c) + ((1 / 2048 * (b + 1)) * weight(a + e))            NEXT a            neuron(c) = 1 / (1 + 2.71828 ^ (-(sum(c) / 1)))            e = e + 784            FOR d = 1 TO 10                sum2(d) = sum2(d) + (neuron(c) * weight2(d + f))            NEXT d            f = f + 10        NEXT c        FOR g = 1 TO 10            neurout(g) = 1 / (1 + 2.71828 ^ (-(sum2(g) / 1)))        NEXT g        FOR z = 1 TO 10            IF (result + 1) = z THEN GOTO 1            IF neurout(result + 1) > neurout(z) THEN count = count + 1            1:        NEXT z        IF count = 9 THEN count2 = count2 + 1        count = 0    NEXT s    CLOSE #1    PRINT count2 / ((s - 1) / 100);    count2 = 0    PRINT epochNEXT epoch'testing the accuracyPRINT "test"count = 0count2 = 0OPEN "mnist.test" FOR BINARY AS #1FOR ss = 1 TO 10000    a\$ = INPUT\$(785, 1)    result\$ = MID\$(a\$, 1, 1)    result = VAL(result\$)    e = 0    f = 0    FOR r = 1 TO 10        sum2(r) = 0        sum(r) = 0        sum(r + 10) = 0        sum(r + 20) = 0    NEXT r    FOR c = 1 TO 30        FOR a = 1 TO 784            b\$ = MID\$(a\$, a + 1, 1)            b = ASC(b\$)            sum(c) = sum(c) + ((1 / 2048 * (b + 1)) * weight(a + e))        NEXT a        neuron(c) = 1 / (1 + 2.71828 ^ (-(sum(c) / 1)))        e = e + 784        FOR d = 1 TO 10            sum2(d) = sum2(d) + (neuron(c) * weight2(d + f))        NEXT d        f = f + 10    NEXT c    FOR g = 1 TO 10        neurout(g) = 1 / (1 + 2.71828 ^ (-(sum2(g) / 1)))    NEXT g    FOR z = 1 TO 10        IF (result + 1) = z THEN GOTO 2        IF neurout(result + 1) > neurout(z) THEN count = count + 1        2:    NEXT z    IF count = 9 THEN count2 = count2 + 1    count = 0NEXT ssPRINT count2 / ((ss - 1) / 100);CLOSE #1`
This is the code to transform the mnist csv from here https://pjreddie.com/projects/mnist-in-csv/into a binary format
Code: [Select]
`OPEN "mnist_train.csv" FOR INPUT AS #1OPEN "mnist" FOR OUTPUT AS #2DO UNTIL EOF(1)    INPUT #1, z    IF z = 0 THEN a\$ = "0" ELSE IF z = 1 THEN a\$ = "1" ELSE IF z = 2 THEN a\$ = "2" ELSE IF z = 3 THEN a\$ = "3" ELSE IF z = 4 THEN a\$ = "4" ELSE IF z = 5 THEN a\$ = "5" ELSE IF z = 6 THEN a\$ = "6" ELSE IF z = 7 THEN a\$ = "7" ELSE IF z = 8 THEN a\$ = "8" ELSE IF z = 9 THEN a\$ = "9"    PRINT #2, a\$;    FOR ww = 1 TO 98        INPUT #1, a, b, c, d, e, f, g, h        PRINT #2, CHR\$(a); CHR\$(b); CHR\$(c); CHR\$(d); CHR\$(e); CHR\$(f); CHR\$(g); CHR\$(h);    NEXT wwLOOPCLOSE #1CLOSE #2`

#### bplus

• Hero Member
•     • Posts: 756
• B = B + _ ##### Re: Artificial neural network
« Reply #1 on: April 09, 2018, 07:00:42 pm »
Hi welcome to forum!

What's a mnist?

https://en.wikipedia.org/wiki/MNIST_database

Oh! You can read the writing on the wall. Will you still love me, will you still need me, when I'm (QB) 64?